Beefy Boxes and Bandwidth Generously Provided by pair Networks
"be consistent"
 
PerlMonks  

In place editing of text files

by jevaly (Sexton)
on May 28, 2009 at 07:50 UTC ( [id://766596]=perlquestion: print w/replies, xml ) Need Help??

jevaly has asked for the wisdom of the Perl Monks concerning the following question:

Dear Monks,
I have a textfile looking like this:
Dit is een voorbeeldtekst The quick brown fox jumps over the lazy dog. SOURCEREPOSITORYNAME Roses are red, violets are blue And Osama Is coming To Kill you

I want to change it in this:
Dit is een voorbeeldtekst The quick brown fox jumps over the lazy dog. SOURCEREPOSITORYNAME=Development Roses are red, violets are blue And Osama Is coming To Kill you

To achieve this epic feat, I wrote the following piece of code (roughly based on my camel):
#!perl -w use strict; my $line; my $CTLfile = 'y:\perl\test.ctl'; local $^I = '.bak'; open (CTLHANDLE, "+<$CTLfile"); while($line=<CTLHANDLE>){ $line =~ s/SOURCEREPOSITORYNAME/SOURCEREPOSITORYNAME=Development/g +; print CTLHANDLE "$line"; } close(CTLHANDLE); undef $^I;

This renders the following output:
Dit is een voorbeeldtekst The quick brown fox jumps over the lazy dog. SOURCEREPOSITORYNAME Roses are red, violets are blue And Osama Is coming To Kill youDit is een voorbeeldtekst Dit is een voorbeeldtekst ver the lazy dog. ver the lazy dog. E E es are red, violets are blue es are red, violets are blue uu

I cannot explain this. I know there are modules helping for in place editing, but I don't want to use them. can someone explain the output, why it looks like this and what I can do to achieve my goal?

Replies are listed 'Best First'.
Re: In place editing of text files
by Corion (Patriarch) on May 28, 2009 at 07:54 UTC

    Reading from a file and in the same moment writing to it is rarely what you want to do. Consider a file to be much like a vinyl record - the read-write head moves forward. If you read a line, the read-write head is now at the end of that line. You then write your modified line to it. This likely is not what you want.

    I recommend either slurping the whole file into memory before modifying, then writing it out as a whole again. Alternatively you can use Tie::File, which allows you to treat a file as an array. The most sane solution would be to use a templating module, like Template Toolkit for example.

Re: In place editing of text files
by svenXY (Deacon) on May 28, 2009 at 08:25 UTC
    Hi,
    perl -pi -e 's/(SOURCEREPOSITORYNAME)/$1=Development/g' y:\perl\test.c +tl
    from perlrun: -i specifies that files processed by the <> construct are to be edited in-place
    svenXY
Re: In place editing of text files
by dHarry (Abbot) on May 28, 2009 at 10:07 UTC

    The -i command-line switch modifies each file in place. It does so by creating a temporary file (but Perl takes care of the details/bookkeeping).

    You can also do this yourself of course:

    open(OLD, "< $old") or die "can't open $old: $!"; open(NEW, "> $new") or die "can't open $new: $!"; while (<OLD>) { # change $_, then... print NEW $_ or die "can't write $new: $!"; } close(OLD) or die "can't close $old: $!"; close(NEW) or die "can't close $new: $!"; rename($old, "$old.orig") or die "can't rename $old to $old.orig: $!"; rename($new, $old) or die "can't rename $new to $old: $!";

    Source Perl Cookbook.

Re: In place editing of text files
by ig (Vicar) on May 28, 2009 at 12:20 UTC

    You are getting the funny output because you are using buffered I/O on the file and you are neither flushing the buffers nor setting the file position when you switch from reading to writing.

    When you read the first line from the file, perl actually reads a buffer full of data. Your file is very small so it fits in the buffer and is read in a single read from the system. Perl then returns the first line from the buffer to your program, leaving the file position at the next character: the start of the second line. The buffer contains the entire file:

    Dit is een voorbeeldtekst The quick brown fox jumps over the lazy dog. SOURCEREPOSITORYNAME Roses are red, violets are blue And Osama Is coming To Kill you
      thanks for the explanation!
      I solved my little problem by just opening the handle two times: one for read and one for write access:
      #!perl -w use strict; my $line; my @array; my $CTLfile = 'y:\perl\test.ctl'; open (CTLHANDLE, "<$CTLfile"); @array = <CTLHANDLE>; close(CTLHANDLE); open (CTLHANDLE, ">$CTLfile"); foreach $line(@array){ $line =~ s/SOURCEREPOSITORYNAME/SOURCEREPOSITORYNAME=Development/g +; print CTLHANDLE "$line"; } close(CTLHANDLE);
      this seems to work fine.

        That will work well for small to moderately sized files: anything that fits in memory, which is quite large these days.It is a good solution in many cases.

        An advantage of writing to a temporary file and renaming is that you are less likely to end up with a truncated or partial file replacing your original. With your method, if your program or the system fails after you open the file for output, which truncates it, and before you finish writing and flushing all your buffers to disk, you will end up with an incomplete file. If you write to a temporary file and only rename it after closing your output file handle the risk of ending up with a truncated file is much much less.

        There is still some risk, even if you write a temporary file, close and rename it. You can reduce the risk further, but not to zero as far as I can tell. Through the various layers of control between your program and the persistent storage medium, it is possible that operations are re-ordered and that abrupt termination (e.g. power loss) can result in corruption despite even extreme measures taken in your program. http://www.gossamer-threads.com/lists/perl/porters/236839 is one writeup on some of these issues.

Re: In place editing of text files
by bibliophile (Prior) on May 28, 2009 at 13:50 UTC
Re: In place editing of text files
by sm@sh (Acolyte) on Jul 22, 2016 at 15:40 UTC
    ok some years have passed, but I don't see File::Slurp mentioned here which provide edit_file and edit_file_lines.
    # replace all occurrences in the file edit_file { s/foo/bar/gs } myfile.txt> # replace occurrences a line at a time edit_file_lines { s/foo/bar/g } myfile.txt>
    that said, I see Tie::File mentioned here which looks like a better solution for my case.
      File::Slurp is no longer recommended. TLDR: It doesn't work well with encodings, is poorly maintained, and its default options are seldom needed.

      ($q=q:Sq=~/;[c](.)(.)/;chr(-||-|5+lengthSq)`"S|oS2"`map{chr |+ord }map{substrSq`S_+|`|}3E|-|`7**2-3:)=~y+S|`+$1,++print+eval$q,q,a,

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://766596]
Approved by Corion
Front-paged by lakshmananindia
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2024-03-29 05:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found