|
Travis M. has asked for the wisdom of the Perl Monks concerning the following question:
Here'a quick update for eveyone who offered help:
First: THANKS!! You guys and gals are great. Your comments and thoughts proved most useful, and helped me think about my problem from a different viewpoint, which ultimately proved to solve my problem. A special thanks to 'beable' who told me about the Tie::File module, which for all intents and purposes saved my bacon. I will note, however, that Perl PPM was less than successful in installing this particular module (as well as a number of others, which leads me to believe that the problem is, perhaps, me... :) )...After much reasearch, however, I was able to install this handy module the archaic way.
In anycase, this may not be the most efficient way of doing this, but it works, and that's all I needed. Here's the code that solved my problem:
tie @array, 'Tie::File', "mdfl.txt";
foreach $line (@update) {
chomp $line;
@bob = split (/\t/, $line);
$dmpiid = $bob[0];
$new = $bob[1];
$ln = "-1";
{
foreach $element (@array) {
$ln ++;
if (($element) =~ (/$dmpiid/)) {
print "Match found at $ln\n";
$slice = $array[$ln];
@fred = split (/\t/, $slice);
$dmpi = $fred[0];
$benum = $fred[1];
$os = $fred[2];
$name = $fred[3];
$lat = $fred[4];
$long = $fred[5];
$last = $fred[6];
$curr = $fred[7];
$last = $curr;
$curr = $new;
$array[$ln] = "$dmpi\t$benum\t$os\t$name\t$lat\t$long\t$last\t$curr";
next;
}
}
}
}
untie @array;
|
Re: flat file databases
by beable (Friar) on Aug 12, 2004 at 05:48 UTC
|
Hello Travis, I'd suggest you take a look at the Tie:File module, which lets you treat files as arrays. This would make it easy for you to change the data in your file, assuming the data were able to be dealt with in this way. Of course, I'm only guessing here, because I haven't seen your data. Here's a simple example of using Tie::File:
#!/usr/bin/perl
use Tie::File;
use strict;
use warnings;
my $filename = "dbfile.txt";
tie my @array, 'Tie::File', $filename or die "can't open $filename: $!
+";
$array[13] = 'blah'; # line 13 of the file is now 'blah'
print $array[42]; # display line 42 of the file
__END__
| [reply]
[d/l] |
|
|
Thank you for your help....I'm now in the process of looking at the tie module...My biggest problem is that each file is read into an array. I have to split each line of each array into yet another array, where I make a an update to one or two particular elements...That's where I get into trouble. I can't seem to make that change propagate back into the very first array (the one from which each line is made into a sub-array).
Here's my code that I am working with. If you get overly bored, would you mind giving it a once-over? I am really at a loss here and have spent several days trying to make it work...
Thanks again,
Travis.
#!C:/Perl/bin/perl -w
#use CGI ':standard';
open (UPDATE, "update.txt") or die "Cannot open update.txt";
@data = <UPDATE>;
close (UPDATE);
#SAMPLE DATA FROM THIS ARRAY IS AS FOLLOWS:
#T00001 0123-12345 DD001 67
#T00002 0123-12345 DD001 99
#T00003 0123-12345 DD002 0
#T00008 0123-12346 DD001 76
#T00014 7777-77777 DD001 88
#T00020 0999-99999 DD001 99
open (MDF, "mdfl.txt") or die "Cannot open update.txt";
@data2 = <MDF>;
@data2 = @data2;
close (MDF);
#SAMPLE DATA FROM THIS ARRAY IS AS FOLLOWS:
#T00001 0123-12345 DD001 100 100
#T00002 0123-12345 DD001 100 100
#T00003 0123-12345 DD001 100 100
#T00004 0123-12345 DD001 100 100
#T00007 0123-12345 DD002 100 100
#T00008 0123-12346 DD001 100 100
#T00013 7777-77777 DD002 100 100
#T00014 7777-77777 DD002 100 100
#T00015 7777-77777 DD003 100 100
#T00016 8888-88888 DD001 100 100
#T00019 8888-88888 DD003 100 100
#T00020 9999-99999 DD004 100 100
#I start by reading each line fo the first array, called @data
#I split this line on tabs
foreach $line (@data) {
chomp $line;
@array = split (/\t/, $line);
$dmpiid = $array[0];
$be = $array[1];
$osuf = $array[2];
$update = $array[3];
#Then I read each line from the second array, splitting it on tabs too
foreach $line2 (@data2)
{
chomp $line2;
@array2 = split (/\t/, $line2);
$dmpiid2 = $array2[0];
$be2 = $array2[1];
$osuf2 = $array2[2];
$last = $array2[3];
$curr = $array2[4];
#next I find where certain values are the same from each array
if (($dmpiid) eq ($dmpiid2)) {
#when found, I update the appropriate values as indicated
$last = $curr;
$curr = $update;
#next, I push the following string back into the @data2 array
$new = "$dmpiid2\t$be2\t$osuf2\t$last\t$curr\n";
push (@data2, $new);
#next I jump out of the inner loop because there's no need to keep ch
+ecking
last;
#however, once I do that, I need to delete the previous instance of
#the data I just updated, otherwise the array will contain both old an
+d
#new data...this is where I get stuck
} #end if
} #end inner foreach
}# end outer foreach
#this is my test which prints my results. unfortunately, the old data
+ is
present allong with the new data.
foreach $item (@data2) {
print "$item\n";
}
| [reply]
[d/l] |
Re: flat file databases
by bobf (Monsignor) on Aug 12, 2004 at 05:06 UTC
|
Updating a file with new data is a common task. Take a look at perlfaq5 (Files and Formats) - the FAQ entitled "How do I change one line in a file" has a chunk of code that might help. | [reply] |
Re: flat file databases
by Zero_Flop (Pilgrim) on Aug 12, 2004 at 05:38 UTC
|
Travis-
How big are these files?
Does the first file only contain one record, or multiple records?
If it has only one record I would try doing this as follows:
Open and read the first file.
Loop though the second file reading one line at a time.
Read the line, compare to the record you want to change to the record you just read, then write to a temp file the correct entry.
Once you have gone though the entire second file. You will have two files, the original second file and a temp file that is identical to the second file but with the record change. If your record change was successful then rename the old second file to something like data.old and rename the temp file to the name of the second file. ( this give you a roll back ability if something goes wrong) You can have multiple data.old files to act as an undo feature.
If you have multiple lines you have to replace, I would consider using a module on CPAN that would give you a generic interface to your data.
Good luck
ZeroFlop
P.S. You may just be off a little in your code. Posting the reivant part will help in supporting you! | [reply] |
|
|
Thanks for the advice, but I am still having problems. I have attached the code I have thus far written (knowing full well it could probably be more concise) as well as some sample data....If you have nothing better to do, would you mind giving it a once over? Perhaps you can provide some additional suggestions...
Thanks again,
Travis
#!C:/Perl/bin/perl -w
#use CGI ':standard';
open (UPDATE, "update.txt") or die "Cannot open update.txt";
@data = <UPDATE>;
close (UPDATE);
#SAMPLE DATA FROM THIS ARRAY IS AS FOLLOWS:
#T00001 0123-12345 DD001 67
#T00002 0123-12345 DD001 99
#T00003 0123-12345 DD002 0
#T00008 0123-12346 DD001 76
#T00014 7777-77777 DD001 88
#T00020 0999-99999 DD001 99
open (MDF, "mdfl.txt") or die "Cannot open update.txt";
@data2 = <MDF>;
@data2 = @data2;
close (MDF);
#SAMPLE DATA FROM THIS ARRAY IS AS FOLLOWS:
#T00001 0123-12345 DD001 100 100
#T00002 0123-12345 DD001 100 100
#T00003 0123-12345 DD001 100 100
#T00004 0123-12345 DD001 100 100
#T00007 0123-12345 DD002 100 100
#T00008 0123-12346 DD001 100 100
#T00013 7777-77777 DD002 100 100
#T00014 7777-77777 DD002 100 100
#T00015 7777-77777 DD003 100 100
#T00016 8888-88888 DD001 100 100
#T00019 8888-88888 DD003 100 100
#T00020 9999-99999 DD004 100 100
#I start by reading each line fo the first array, called @data
#I split this line on tabs
foreach $line (@data) {
chomp $line;
@array = split (/\t/, $line);
$dmpiid = $array[0];
$be = $array1;
$osuf = $array2;
$update = $array3;
#Then I read each line from the second array, splitting it on tabs too
foreach $line2 (@data2)
{
chomp $line2;
@array2 = split (/\t/, $line2);
$dmpiid2 = $array2[0];
$be2 = $array21;
$osuf2 = $array22;
$last = $array23;
$curr = $array24;
#next I find where certain values are the same from each array
if (($dmpiid) eq ($dmpiid2)) {
#when found, I update the appropriate values as indicated
$last = $curr;
$curr = $update;
#next, I push the following string back into the @data2 array
$new =
"$dmpiid2\t$be2\t$osuf2\t$last\t$curr\n";
push (@data2, $new);
#next I jump out of the inner loop because there's no need to keep
checking
last;
#however, once I do that, I need to delete the previous instance of
#the data I just updated, otherwise the array will contain both old and
#new data...this is where I get stuck
} #end if
} #end inner foreach
}# end outer foreach
#this is my test which prints my results. unfortunately, the old data is
present allong with the new data.
foreach $item (@data2) {
print "$item\n";
}
| [reply] |
|
|
This may not be a popular solution, but if your files are a decent size you could use a hash.
Read each line of the data file and split on the first tab.
Turn this into a hash with the ID as the key and the remaining of the line as the data for the hash. (You could also use a hash of arrays)
Now you can load the update file, line by line.
Turn the line into an array.
Use the first array entry as the hash key to get the hash data.
Create the new hash data string from the update data.
Update the Data Hash.
Read next line from update file.
You end up with an updated hash. Now print that back to the file.
| [reply] |
Re: flat file databases
by McMahon (Chaplain) on Aug 12, 2004 at 14:34 UTC
|
Certainly code would allow more explicit help; but you might want to investigate the DBD::CSV or DBD::AnyData modules. They allow you treat all sorts of data as a database and to perform SQL operations on those data.
I've been working with DBD::CSV a lot recently. It's a very handy tool to have. Even if it doesn't solve your immediate problem, I think you might want to know about it anyway. | [reply] |
Re: flat file databases
by Plankton (Vicar) on Aug 12, 2004 at 14:49 UTC
|
I think you should get rid of the flat-file databse all together and use SQLite instead. You'll be much happier in the long run.
| Plankton: 1% Evil, 99% Hot Gas. |
| [reply] |
Re: flat file databases
by wfsp (Abbot) on Aug 14, 2004 at 07:39 UTC
|
Here's my go:
It's written in long hand with lots of variables to help (me) 'see' the method.
It loads the updates into a hash then loops through the data one line at a time making changes when the id matches and writing everything to a new file.
Any use?
#!/bin/perl5
use strict;
use warnings;
my %update;
open (UPDATE, "update.txt") or die "Cannot open update.txt";
while (<UPDATE>){
chomp;
# T00001 0123-12345 DD001 67
my @array = split '\t';
my $key = $array[0];
$update{$key} = [ @array[1..3] ]
}
close (UPDATE);
open (MDF, 'mdfl.txt') or die "Cannot open update.txt";
open (OUT, '>', 'out.txt') or die "Cannot open out.txt";
while (<MDF>){
chomp;
my $record = $_;
my @fields = split '\t', $record;
my $dmpiid = $fields[0];
if ( exists $update{$dmpiid} ){
my $curr = $fields[4];
my $update = $update{$dmpiid}[2];
$fields[3] = $curr;
$fields[4] = $update;
$record = join "\t", @fields;
}
print OUT $record, "\n";
}
close (MDF);
close (OUT);
| [reply]
[d/l] |
Re: flat file databases
by neilh (Pilgrim) on Aug 12, 2004 at 04:38 UTC
|
Hi Travis,
If you are opening the flat file db, you can use the $_ special variable - detailed here
Simply replace the current contents of $_ with the contents that you want to put in there.
Some example code/data would probably help here.
Neil | [reply] |
|
|
I'm sorry, but I don't think this is accurate at all. Unless you've somehow aliased $_ to something tied to the file that handles reading and writing, this will not work automagically.
Example code would be nice.
| [reply]
[d/l] |
|
|
I'm talking about something like
*Poor code stripped*
This is how I tend to do things.
Neil
Update
I completely stuffed this one.. I missed all sorts of things like writing back to the file, opening the file, etc.
Sorry if I caused any confusion.
Neil
| [reply]
[d/l] |
|
|
