Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Should Modules Do I/O?

by BrowserUk (Patriarch)
on Mar 18, 2005 at 18:07 UTC ( [id://440760]=note: print w/replies, xml ) Need Help??


in reply to Should Modules Do I/O?

I think you asked a very good question and are thinking along the right lines.

As counterpoint to other responses, I hate modules that insist on writing stuff to disk, and force me to re-open the output file* in order to get the data back into my program.

Maybe

  • I only need a 1 of the 100,000 files in the archive.
  • I don't have enough disk space to expand the whole archive and want to process the files one at a time.
  • I want to change the names of the files before they are written.
  • Or only write those that contain a given string.
  • Or I want write 'head' and/ or 'tail' utilities to apply to the files inside the archive.
  • Or I want to process a huge archive from an ftp stream and stop transferring when I find the file I want.
  • I need to create the file with special permissions or security attributes, or on a different machine.
  • Or...

So, I'd infinitely prefer an interface that allowed me to supply an open filehandle for the archive file (new or existing), and methods for adding and retrieving files from the archive:

Reading

use Your::Module; open my $arc, 'ftp ftp://some.dot.com/pub/3GB.archive |' or die; local $/ = \65536; ## Could your module handle this? my $arcObj = Your::Module->new( $arc ); while( my( $name, $dataRef ) = $arcObj->next ) { if( $name =~ m[^file(\d+.type)$] and $$dataRef =~ m[this|that] ){ open $out, '>', localtime . $1 or die $!; print $out $$dataRef; last } }

Writing

use Win32API::File qw[ :all ]; use Your::Module; my $hObject = CreateFile( '//?/UNC/Server/Share/Dir/File.Ext', FILE_READ_EA, FILE_SHARE_READ, pack( "L P i", 12, $pSecDesc, $bInheritHandle ), TRUNCATE_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN|FILE_FLAG_WRITE_THROUGH, SECURITY_IDENTIFICATION|SECURITY_IMPERSONATION, 0 ) or die $^E; OsFHandleOpen( FILE, $hObject, $sMode ) or die $^E; my $arcObj = Your::Module->new( \*FILE ); opendir DIR, '//SERVER/DIR/'; while( my $file = readdir DIR ) { open my $fh, '<', $file or die $!; $arcObj->addFile( "/DIR/$file", do{ local $/; <$fh> } ); } close DIR;

Providing that kind of flexibility for the users, combined with the reduction in code in your module, would make your module more powerful and useful.

Especially as you're providing 'arcit.pl' and 'unarcit.pl' scripts for the simple case, thereby avoiding the "boilerplate code" charge.

(*usually after patching the module to provide a way of finding out the filename)


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
Lingua non convalesco, consenesco et abolesco.
Rule 1 has a caveat! -- Who broke the cabal?

Replies are listed 'Best First'.
Re^2: Should Modules Do I/O?
by Mugatu (Monk) on Mar 18, 2005 at 18:54 UTC
    I hate modules that insist on writing stuff to disk, and force me to re-open the output file in order to get the data back into my program

    That's why other Monks suggested that both interfaces be supported. Your post would make it three. But that's fine, as long as each one is documented and maintained. Choice is good. False dilemmas are bad.

      both interfaces be supported

      Providing both seems like YALORC to me. Yet Another Lump Of Reduntant Code,

      Your post would make it three

      With the interface I describe, the other two interfaces can be trivially derived through subclassing, or just simple procedural wrappers.

      That could be done as a part of the module, but I see no value-add in that, as it is equally trivial for the user To Do It themselves, and they can tailor it to their exact requirements, instead of having to work around the supplied interface.

      Neither of the other two interfaces can be easily wrapped to provide each other, nor that which I described.


      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      Lingua non convalesco, consenesco et abolesco.
      Rule 1 has a caveat! -- Who broke the cabal?

        As you acknowledge, these three interfaces can be implemented using eachother. The higher level methods can use the lower level methods. There is nothing redundant there.

        Nobody is suggesting that the OP should make one and only one interface available, except you (unless I am misunderstanding your position). Few users will have to "work around the supplied interface" if the interface embodies the common usage scenarios. That is the value-add.

        sub get_file { my $self = shift; my $filename = shift; my $outputdir = shift; my $ofh = IO::File->new(File::Spec->catfile($outputdir, $filename), +'w'); $self->get_filehandle($ofh); }
        The amount of code I get to save by having this in the module is awesome. And, thus, the number of bugs I'll have in this code will be darned few. Approaching zero the second time I use it.

        Redundant is writing the above 12 times rather than having a single sub doing it for me.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://440760]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others learning in the Monastery: (3)
As of 2024-04-25 10:08 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found