Thanks for the link. I am sure I can use that module in a script I wrote at work, but not here. Here, I may be comparing two large tables and finding one or two rows that differ. I can't put it all in memory, especially not in seen-hashes. I need to be able to suck rows one at a time from MySQL, in sorted order, and have Perl know when a row is missing from one table because one row is "greater than" or "less than" the other.
I've seen the term XY but not seen a definition. What does it mean?
| [reply] |
Generally an XY question is one that pertains to solving a detail problem without providing the larger context. Very often in such a case the larger context suggests a better solution than solving the detail problem.
I don't see a better solution straight off for your problem here however, although the larger context that you've provided may help those more database savvy than I to find a good solution for you.
DWIM is Perl's answer to Gödel
| [reply] |
xaprb:
Perhaps the right approach would be to use a mergesort-like method. Basically, execute a select on each table, specifying the order so MySQL will create the proper order for you. Then you can process the tables in parallel. To see an example, I previously described a similar thing in Re: How to deal with Huge data.
...roboticus
| [reply] |
Thanks! This is *exactly* what I'm doing currently. The only trouble is to get the le, ge, cmp etc operators to agree with MySQL's idea of whether a row is lt/gt/eq the other row.
Imagine the left-hand table has "éclair" and "ecstatic", and the right-hand table has only "ecstatic". MySQL sorts é before other 'e' values, so the merge algorithm sees "éclair" on the left and "ecstatic" on the right after fetching the first row from each table. Perl thinks "ecstatic" should come first though -- at least that's how it works with just the default collations. So the merge algorithm concludes "ecstatic" doesn't exist in the left-hand table. After this, it runs out of rows in the right-hand table and decides "éclair" doesn't exist there, then fetches the next row from the left-hand table -- and decides "ecstatic" doesn't exist in the right-hand table.
It's quite a train wreck unless I get Perl's sorting to exactly match MySQL's!
| [reply] |
