Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Script fails to insert text and appends it towards the end of file.

by always_coys (Novice)
on Nov 21, 2020 at 12:40 UTC ( [id://11123966]=perlquestion: print w/replies, xml ) Need Help??

always_coys has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks,

I am trying to automate the running of a finite element program using a Perl script. It requires me to copy a block of text (each line starting with the same keyword) and insert it to a second file, at a particular line number. There is text in the second file, before and after this line number. With help from the Monks, I was able to update my script. However, I am noticing that the script is simply appending to the output file, rather than inserting the block of text to the desired line number.

The input file is given below. I am trying to copy the lines beginning with the keyword NODE and inserting it in the output file, at line number 37. Basically, I am trying to append to a section of the output file (to lines containing the same keyword). I have left sufficient space in the output file for inserting text from the input file.

NODE 32 0.00000 0.00000 -1.90000 NODE 33 0.00000 0.00000 -5.50000 NODE 34 0.00000 0.00000 -9.00000 NODE 35 0.00000 0.00000 -15.00000 NODE 36 0.00000 0.00000 -18.90000 NODE 37 0.00000 0.00000 -22.40000 NODE 38 0.00000 0.00000 -25.90000 NODE 39 0.00000 0.00000 -29.00000 NODE 40 0.00000 0.00000 -32.50000 NODE 41 0.00000 0.00000 -33.90000 NODE 42 0.00000 0.00000 -62.90000 BEAM 26 27 26 1 14 1 BEAM 27 28 27 1 13 1 BEAM 28 29 28 1 12 1 BEAM 29 30 29 1 11 1

The output file is given below.

'Node ID X Y Z BC NODE 1 4.51000 0.00000 79.00000 NODE 2 0.00000 0.00000 79.00000 NODE 3 0.00000 0.00000 78.27000 NODE 4 -1.88000 0.00000 78.27000 NODE 5 0.00000 0.00000 76.80000 NODE 6 0.00000 0.00000 74.46000 NODE 7 0.00000 0.00000 71.66000 NODE 8 0.00000 0.00000 68.86000 NODE 9 0.00000 0.00000 66.07000 'Elem ID np1 np2 material geom lcoor ecc1 BEAM 1 2 1 2 36 2 BEAM 2 3 2 2 36 1 BEAM 3 3 4 2 36 2 BEAM 4 5 3 2 36 1 BEAM 5 6 5 3 35 1 BEAM 6 7 6 3 34 1 PIPE 19 4.489 0.022 PIPE 20 4.488 0.021 PIPE 21 4.487 0.020 PIPE 22 4.395 0.018 PIPE 23 4.351 0.018 PIPE 24 4.261 0.017

My script below, should ideally place the block of text beginning with NODE from the input file, on line number 11 of the output file (i.e. just below another block of text that begins with NODE. However, my script is simply appending the text to the output file (after counting 11 lines from the end). Could the Monks please help me identify what went wrong?) Thanks a lot (always_coys).

use strict; use warnings; # use the Tie::File option use Tie::File; my @records; tie @records, 'Tie::File', "output.fem"; my $in_file = "fake_vals.fem"; my $outIndex = 11; # use three parameter open '<' - read mode open my $in, '<', "$in_file" or die "cannot open '$in_file' for readin +g: $!\n"; while (my $line = <$in>) { next if $line !~ /\bNODE\b/i; chomp $line; $records[$outIndex] = $line; ++$outIndex; }

Replies are listed 'Best First'.
Re: Script fails to insert text and appends it towards the end of file.
by haukex (Archbishop) on Nov 21, 2020 at 13:26 UTC
    However, my script is simply appending the text to the output file (after counting 11 lines from the end).

    I initially couldn't reproduce this, but managed when running the script on Windows and the files having LF (not CRLF) line endings. Using tie @records, 'Tie::File', "output.fem", recsep=>"\n"; makes it work again. (See also the module's _default_recsep.)

    However, I would not implement this script this way, since it requires you to know the correct place to insert the lines beforehand, and it requires there to be enough space for the lines to be inserted, otherwise data will be overwritten. The following uses a state machine pattern, and has the advantages that it does a bit of verification on the input file (it checks that there's only one section of NODE lines) and that it doesn't care how many lines both the input files have. Also, it won't have the above issue, because normal opens on Windows will normally do the right thing with line endings. Plus Tie::File adds significant overhead to the program.

    (Update: Shameless plug: If you want to overwrite the input file, then see my module File::Replace for an easier/safer way to do so.)

    use warnings; use strict; my $DATAFILE = 'data.txt'; my $INSERTFILE = 'insert.txt'; my $OUTFILE = 'output.txt'; open my $ofh, '>', $OUTFILE or die "$OUTFILE: $!"; select($ofh); # "print"s will go to this handle now open my $dfh, '<', $DATAFILE or die "$DATAFILE: $!"; my $state = 'before nodes'; while ( my $line = <$dfh> ) { if ( $line =~ /^\s*NODE\b/ ) { if ( $state eq 'before nodes' ) { $state = 'in nodes'; } elsif ( $state eq 'in nodes' ) { } elsif ( $state eq 'after nodes' ) { die "expected only one section of NODEs" } else { die $state } } else { if ( $state eq 'before nodes' ) { } elsif ( $state eq 'in nodes' ) { do_node_insert($INSERTFILE); $state = 'after nodes'; } elsif ( $state eq 'after nodes' ) { } else { die $state } } print $line; } close $dfh; sub do_node_insert { my $file = shift; open my $ifh, '<', $file or die "$file: $!"; while ( my $line = <$ifh> ) { next unless $line =~ /^\s*NODE\b/; print $line; } close $ifh; }

      Dear haukex

      Thanks a lot for the wonderful explanation. Adding recsep=>"\n" has solved my problem and this is enough, in the first pass (as I know the line numbers to insert the text, beforehand). However, eventually I intent to use the state machine pattern, or to overwrite the file in each iteration. Thank you for the pointers and for thinking way ahead of me! always_coys.
Re: Script fails to insert text and appends it towards the end of file.
by bliako (Monsignor) on Nov 21, 2020 at 13:19 UTC

    It worked for me, but it inserted the block into line 12 (as $array11 refers to its 12th item i guess?):

    'Node ID X Y Z BC NODE 1 4.51000 0.00000 79.00000 NODE 2 0.00000 0.00000 79.00000 NODE 3 0.00000 0.00000 78.27000 NODE 4 -1.88000 0.00000 78.27000 NODE 5 0.00000 0.00000 76.80000 NODE 6 0.00000 0.00000 74.46000 NODE 7 0.00000 0.00000 71.66000 NODE 8 0.00000 0.00000 68.86000 NODE 9 0.00000 0.00000 66.07000 NODE 32 0.00000 0.00000 -1.90000 NODE 33 0.00000 0.00000 -5.50000 NODE 34 0.00000 0.00000 -9.00000 NODE 35 0.00000 0.00000 -15.00000 NODE 36 0.00000 0.00000 -18.90000 NODE 37 0.00000 0.00000 -22.40000 NODE 38 0.00000 0.00000 -25.90000 NODE 39 0.00000 0.00000 -29.00000 NODE 40 0.00000 0.00000 -32.50000 NODE 41 0.00000 0.00000 -33.90000 NODE 42 0.00000 0.00000 -62.90000 'Elem ID np1 np2 material geom lcoor ecc1 BEAM 1 2 1 2 36 2 BEAM 2 3 2 2 36 1 BEAM 3 3 4 2 36 2 BEAM 4 5 3 2 36 1 BEAM 5 6 5 3 35 1 BEAM 6 7 6 3 34 1 PIPE 19 4.489 0.022 PIPE 20 4.488 0.021 PIPE 21 4.487 0.020 PIPE 22 4.395 0.018 PIPE 23 4.351 0.018 PIPE 24 4.261 0.017
      Thanks, bliako, the suggestion from haukex worked out for me!
Re: Script fails to insert text and appends it towards the end of file.
by GrandFather (Saint) on Nov 22, 2020 at 08:24 UTC

    It's kinda hard to tell where this code may be going, but it smells like there should be a database involved. If this sort of random access to lines continues to be required beyond an initialisation phase then a database is likely to perform better with less messing around and less effort than most other techniques.

    Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond
      Hi GrandFather, there is indeed a database involved. I am using Perl to run multiple instances of a finite element software. So, it requires the input files to be updated for every run, using values from the database.

        In that case wouldn't it be more efficient to add a table or tables to the database rather than work with external temporary files?

        Optimising for fewest key strokes only makes sense transmitting to Pluto or beyond
Re: Script fails to insert text and appends it towards the end of file.
by Bod (Parson) on Nov 22, 2020 at 21:15 UTC

    Why not simply splice the lines from the input file into an array with your output file?

    use strict; my $infile = 'input.txt'; my $otfile = 'output.txt'; my $token; our @outfile; # Read existing output file into array open my $ot, '<', $otfile or die "$otfile: $!\n"; while (<$ot>) { push @outfile, $_; } close $ot; # Iterate over input file appending rows to array open my $in, '<', $infile or die "$infile: $!\n"; while (<$in>) { (undef, $token, undef) = split /\W/, $_; &insert($token, $_) if $token; } close $in; # Replace output file eliminating blank lines open $ot, '>', $otfile or die "$otfile: $!\n"; foreach my $line(@outfile) { print $ot $line if $line !~ /^ ?\n/; } exit 0; # Insert the line into @outfile at the correct place sub insert { my $token; my $flag = 0; for (my $i; $i < @outfile; $i++) { (undef, $token, undef) = split /\W/, $outfile[$i]; $flag = 1 if $token eq $_[0]; if ($flag and $token ne $_[0]) { print "$_[0]\n"; splice @outfile, $i, 0, $_[1]; $flag = 0; } } }

    I wasn't sure if you wanted to insert lines starting with BEAM into the output file after the BEAM entries already there but the script above does this. Whatever is in the input file, it adds those to the output file immediately after entries with the same initial 'token'.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11123966]
Approved by marto
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others having a coffee break in the Monastery: (7)
As of 2024-03-28 10:30 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found