Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re^4: Text Analysis Tools to compare Slinker and Stinker?

by mojotoad (Monsignor)
on Jan 22, 2003 at 06:57 UTC ( [id://228957]=note: print w/replies, xml ) Need Help??


in reply to Re: Re: Re: Text Analysis Tools to compare Slinker and Stinker?
in thread Text Analysis Tools to compare Slinker and Stinker?

'Misspellings' are precisely where Bayesian filtering, once trained, will help tremendously (though as others have pointed out, never conclusively).

As an example from the anti-spam efforts, once Bayesian filtering was enabled they were amazed that single token with the highest probability of indicating spam was 'FF0000', the hex value for bright red. Unexpected, but damning.

Consistently misspellt words could show up accordingly.

Mattt

  • Comment on Re^4: Text Analysis Tools to compare Slinker and Stinker?

Replies are listed 'Best First'.
Re: Re^4: Text Analysis Tools to compare Slinker and Stinker?
by castaway (Parson) on Jan 22, 2003 at 08:23 UTC
    Hmm, I don't know about the rest of you, but when I'm typing a lot, I generally don't make the same mistakes. Ok, when someone consistently misspells the same word, its probably because they don't know how to spell it, but what I'm getting at is, that I more often than not just hit the keys in the wrong order.
    When there's a lot going on in a forum (or Mud, whatever) then one tends to get out as much as possible in order to keep up, which produces a lot of inconsistent misspellings. (It does in my case anyway, I'd be glad to prove it).
    Which makes me think that comparing misspellings is not such a good way to do it.

    Example: German word 'erzaehl', comes out as 'erzahle', 'erzeahl', 'erzaelh' etc. (I've even made aliases for most of these ;)

    C.

[reply]

In Section Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: note [id://228957]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others avoiding work at the Monastery: (4)
As of 2024-04-25 02:24 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found