Hey All,
I'm trying to find a way to calculate the total number of matching words in a bunch of strings. I have a database with thousands of strings (and unique ID nums), and I can't just do the obvious brute force solution, as it takes WAY too long (see below). Basically I wanna end up with an array that tells me which strings have high probabilities of being the same string (perhaps the same information, told multiple ways). EX:
The quick brown fox jumped over the lazy dogs.
vs.
The quick brown dogs jumped over the lazy fox.
Both strings say the same thing, but they would not generate a normal ($string1 eq $string2) match. The trick is, I also want to be told that I have a 'near' match on a string (perhaps "the very quick brown...") too. The data currently exists in a two demsional array:
while (@row=$sth->fetchrow_array())
{
$big_deal[$count][0]= $row[0]; # Assign Unique ID to element 0
$big_deal[$count][1]= $row[1]; # Assign string to element 1
}
print ($#big_deal+1) . " records found.<br>\n";
I played with this all morning, before it dawned on me that this might be something someone else has done, or would know of a better way to do then I. My solution currently involves six loops (shudder) and a search of only 200 strings returns a result in just under two hours (on a decently fast Xserve). Any advice appriciated!
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|