Beefy Boxes and Bandwidth Generously Provided by pair Networks
P is for Practical

Casual Communication, AI, and Text Processing

by one4k4 (Hermit)
on Mar 28, 2002 at 16:21 UTC ( #155023=perlquestion: print w/replies, xml ) Need Help??

one4k4 has asked for the wisdom of the Perl Monks concerning the following question:

Tis been a while since I've posted, especially in SoPW. I thought up an idea while thinking about the smarterchild AIM bot. Currently this bot has somewhat "canned" responses (imho) to your statements. I thought about that bot, and the everything project and wondered if I could create a bot that would chat with it's users and ask for help in expanding it's knowledge base. Somewhat of a dictionary, with definitions submitted by the chatters.

This good and all, but what about responces to it's questions, and processing the actual words in the statements? This node (AI -- Artificial Intelligence) helped me quite a bit, but I'm not sure where to start.

It's almost as if I want to have a set of "control" words, and take words in question and compare them to the control words. A table of synonyms, antonyms (spelling mistakes..) and rate the possible control words against the results.

So if you said "My dog is a real pain" it would take "dog" and compare that to things like "animal" and then it would know that the subject of the sentence is animals, in reality "my dog". Basically, applying english language rules to text processing with perl. Yeah, that about summs it up.

Knowing "dog" is the noun and subject of the sentence, if it does not know what dog is, it would ask..

Am I crazy? Am I thinking about something that is too far fetched? Am I entering a good part of the perl community? I know this would at bare minimum help with my OOP and the like..

Welp, that about summs it up friends. Feel free to shoot me in the foot if I haven't already, but any help or insight is appreciated.

I'm only thinking about this because jmcnamara told me I couldn't help him with Spreadsheet::WriteExcel until he gets back on track. ;)

_14k4 - (
  • Comment on Casual Communication, AI, and Text Processing

Replies are listed 'Best First'.
Re: Casual Communication, AI, and Text Processing
by dragonchild (Archbishop) on Mar 28, 2002 at 16:48 UTC
    Basically, you're trying to quick-parse sentences. My cousin did her master's thesis on comparing two algorithms for natural language parsing. Just two, and she got a Masters.

    The problem is that the rules for parsing a natural language are context-driven. In your example, "my dog" is the subject because it precedes "is", not because it's a noun. (And, just to make things more complicated, "dog" is considered a noun solely due to placement. There is a verb "dog", too.)

    I'll give you another example. "Pole, you Pole!" That is a perfectly good imperative sentence in english. It breaks down as "Verb (command), Object". However, if I change it to "Pole, you pole!" ... That breaks down to "Object, Verb (command)". The only difference was capitalization. But, I suspect all the english-speakers could understand what I was saying. How would you encapsulate those rules?

    I'm not trying to say it can't be done. In fact, I'm working on an algorithm myself, in my spare time. However, I certainly do not think that Perl is the place to do it. The data structures are too large and the algorithms too computationally-intensive for Perl to be a good language.

    We are the carpenters and bricklayers of the Information Age.

    Don't go borrowing trouble. For programmers, this means Worry only about what you need to implement.

      Hrm, I like your commentary. Very interesting indeed. I was thinking of maybe something similar to the concept behind soundex in the SQL langauge. Even doing the "soundex" like calculations on a table level? But my interest is indeed sparked, and I shall go from here.

      I did, however, create a very basic sourceforge project for this idea located at Expect things to change. And change often. Sourceforge is basically there for CVS and portability.

      Thank you so much for your thoughts, they're a *big* help.

      _14k4 - (
Re: Casual Communication, AI, and Text Processing
by lachoy (Parson) on Mar 28, 2002 at 16:43 UTC
Re: Casual Communication, AI, and Text Processing
by trs80 (Priest) on Mar 28, 2002 at 17:05 UTC
    Most anyone attempting to create AI has to be crazy. The complexity surrounding language and its use is a mind numbing thing to consider. There was a very good article in Harper's April 2001 called "Tense Present". I think what you are trying to accomplish really needs to be throughly broken down into the parts that make up languages, the more the better, and discern the commonality between them.

    There are good articles on Perl for language parsing in TPJ. Fall 2000 and Summer 2000 jump to my mind immediately.

    Take a look at the Lingua related modules to see what others have done.
      Take a look at the Lingua related modules to see what others have done.

      Probably Lingua::LinkParser could be a good starting point. There is also a TPJ article by Dan Brian about that Module.

        I just wanted to say, that your link and quick comment deserves more ++'s than I can give. That module will help me a lot. If anything to keep my mind busy for the day. :) Thank you!

        _14k4 - (
Re: Casual Communication, AI, and Text Processing
by RMGir (Prior) on Mar 28, 2002 at 16:43 UTC
    A quick google search turned up BotSpot, which looks like a good site for investigating this kind of question.

Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://155023]
Approved by root
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others surveying the Monastery: (2)
As of 2023-06-07 03:18 GMT
Find Nodes?
    Voting Booth?
    How often do you go to conferences?

    Results (29 votes). Check out past polls.