Beefy Boxes and Bandwidth Generously Provided by pair Networks
Think about Loose Coupling
 
PerlMonks  

Re: What are the monks doing with Perl and Linguistics?

by matsmats (Monk)
on May 05, 2003 at 17:25 UTC ( [id://255674]=note: print w/replies, xml ) Need Help??


in reply to What are the monks doing with Perl and Linguistics?

Funny that you should ask this today. I was just about to look around the monastery for the same.

I'm just now finalizing a system to automatically determine if general news articles in a large newsfeed is 'bad news or good news'. i.e. to mark negative events, if its negative stock market reports, a local sports team losing, negative criticism, crime etc.

Perl has been excellent for implementing such a system and I find the tools at CPAN invaluable, but as I only work with Scandinavian languages a lot of the modules are not available to me as they are a too specific to English. I would guess that the stemmers are a basic tool for everyone doing computational linguistics, though, and they are available for all the languages I need them for.

Mats Stafseng Einarsen

Replies are listed 'Best First'.
Re: Re: What are the monks doing with Perl and Linguistics?
by allolex (Curate) on May 05, 2003 at 19:32 UTC

    Sounds like interesting stuff.

    Jon Kleinberg at Cornell is doing some research into topic "burst" identification in real-time text data streams. You can have a look at some of his preliminary results here. It might be interesting to compare whether your results match his. His stuff is all for English (of course).

    I'd be curious to hear about how your doing your identification---I'm assuming a kind of basic separation of lexemes into categories (positive, negative, neutral), or do you have a level of abtraction there. If you'd like to share, you can reach me via e-mail from my homepage or /msg me.

    Where the language modules are concerned, I really feel for you. Due to time constraints, we have kludged together a sentencizer for French because there was not one readily available to suit our needs, but due to everything else being prioritized above it, it is utter crap (does the job mostly right, but with unpredictable results).

    --
    Allolex

      I've kludged together a sentencizer for Norwegian (based on Text::Sentence) myself, so I know what you mean. It's not that bad, but the fact that some people are working at doing the same for audio streams is rather humbling.

      Kleinbergs work seems interesting. I vaguely remember reading about it in the news, so I look forward to looking into it.

      The negative/positive identifier is company work, so I'll probably be better of not elaborating on it for now.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://255674]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chanting in the Monastery: (4)
As of 2024-03-19 05:09 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found