http://qs321.pair.com?node_id=322519

AcidHawk has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I am trying to read a dir for small files on one machine and if a file exists transmit it over a socket I have created to another machine. These files are typically small xml files but coule also be small zip files. (I still need to put in checking for large files to ignore them.) The problem I have is that the first files goes over and gets saved successfully BUT the next file is not correct. The filename and size and first line of the next file is sent over in one step. So my output looks as follows 

Got: X-Call~4.xml#:#380_<?xml version="1.0"?>

[download]
instead of 
Got: X-Call~4.xml#:#
Got:380_
Got:<?xml version="1.0"?>

[download]

So this is what I have...

Server... 

#!/usr/bin/perl

use strict;
use warnings;

use IO::Socket;
use IO::Select;
use Time::Local;
use File::Basename;

######################################################################
+##################
#xml Variables
my ($xinput_path, $xmlout, $debug, $port, $logfile, $new_file, $rules)
+;
my ($hb, $hb_prev, $now, $hb_email, $hb_duration, $buffer, $max_client
+s, $slm);
my ($checkdir_thresh, $checkdir_duration, $checkdir_prev, $manager);
#Socket Variables
my ($remote, $sock, @client_list, $sel, $new_remote);
#Misc Variables
my ($VERSION);
######################################################################
+##################
my $rc = 0;
$VERSION     = "1.0.0";
my $timeout    = 5;
my $save_dir = './files' ;

unless (&Create_Socket()) {
    print "Problem with Creating a Socket: $!\n";
    exit(99);
}

my ($filesave, $filesize, $filename, $data_content);


while(1) {
    my @client_list = $sel->can_read(1);
    foreach $remote(@client_list) {
        if ($remote == $sock) {
            $new_remote = $sock->accept;
            $sel->add($new_remote) if ($new_remote);
        }
        else {
            my $in;
            if (sysread($remote, $in, 1024)) {
                if (defined $remote) {
                    print "Got: $in\n";
                    if ($in =~ /#:#$/) {
                        $filename = $in;
                        $filename =~ s/#:#$// ;
                        print "Filename = $filename\n";
                    }
                    elsif ($in =~ /_$/) {
                        $filesize = $in;
                        $filesize =~ s/_$// ;
                        print "Filename = $filename\n";
                        print "Filesize = $filesize\n";
                    }
                    elsif ($in =~ /__END$/) {
                        close (FILENEW) ;
                        print "...\n\n" ;
                        $sel->remove($remote);
                        close($remote);
                        ($filesave, $filesize, $filename, $data_conten
+t) = 0;
                    }
                    else {
                        print "About to save file\n";
                        unless ($filesave) {
                            $filesave = "$save_dir/$filename" ;
                            print "FileSave - $filesave\n";
                            if (-e $filesave) {
                                unlink ($filesave) ;
                            }
                            print "Saving: $filesave ($filesize bytes)
+\n" ;
                        }
                        open (FILENEW,">>$filesave") ;
                        binmode(FILENEW) ;
                        print FILENEW $in ;
                    }
                }
                else {
                    print "DEBUG       - \$remote Client NOT defined\n
+";
                }
            }
            else {
                print "WARNING     - X-InputServer Could not Read data
+ from a connected socket: @_\n";
                $sel->remove($remote);
                close($remote);
            }
        }
    }
}

exit($rc);

######################################################################
+##################
##
##        SUB ROUTINES
##
######################################################################
+##################
#Create a Socket to listen on
sub Create_Socket
{
    $SIG{CHLD} = 'IGNORE';

    $sock = IO::Socket::INET->new(
            LocalPort => 6123,
            Proto => 'tcp',
            Reuse => 1,
            Listen => 100
            ) or return(0);

    $sel = IO::Select->new($sock);

    print "INFORMATION - Socket Created ... Successful\n";
    print "INFORMATION - Awaiting messages on: 6123\n";
    return(1);
}

[download]
Client... 
#!/usr/bin/perl

use strict;
use warnings;
use IO::Socket;        #Socket Handeling Functions
use File::Basename;    #To handle the path and filename functions

######################################################################
+##################
#program version
my $VERSION="4.0.0";
#Setup Vars
my ($xbridge_path, $sock);

my $rc        = 0;
my $port    = "6123";
my $srv        = "192.x.x.x";
######################################################################
+##################

if (&Setup_Log()){
    while(1) {
        my @list = ();
        if (opendir DIR, "./files") {
            while (my $file = readdir DIR) {
                next if $file =~/^\./;
                push @list, $file;
            }
        }
        if ($list[$#list]) {
            print "about to process ./files/$list[$#list]\n";
            if (&Transmit("./files/$list[0]")) {
                print "Moved ./files/$list[0]\n";
                unlink("./files/$list[0]");
            }
        }
        else {
            print "Nothing to process yet\n";
            sleep(1);
        }
    }
}
else {
    print "X-Bridge Could Not Perform the Setup_Log: $!";
    $rc = 1;
}

exit ($rc);

######################################################################
+##################
##
##        SUB ROUTINES
##
######################################################################
+##################
#Brief Setup_Log to initiate the LOG file
sub Setup_Log
{
    my $rc = 0;

#    Open Config File and Read Values into Hash
     my $path = "$ENV{'MyConfigDir'}/XInput";

    if ($path =~ /:/) {
        if ($xbridge_path = Win32::GetShortPathName($path)) {
            $xbridge_path =~s /\\/\//g;
            print "path = $path\n";
            $rc = 1;
        }
        else {
            print "$0 - Path Problems in Setup for $path: $^E\n";
        }
    }
    else {
        print "path = $path\n";
        $rc = 1;
    }

    return($rc);
}

sub Transmit
{
    my $rc            = 0;
    my $bandwidth        = 1024*5 ; # 5Kb/s
    my $file        = shift;

    if (-s $file) {
        my $file_size     = -s $file ;
        my ($name, $path, $type) = fileparse($file, '\..+');
        my $file_name = "$name$type";
        print "Filename = $file_name\n";
        if ($sock = new IO::Socket::INET (
                                     PeerAddr => $srv,
                                     PeerPort => $port,
                                     Proto => 'tcp',
                                    )) {
            $sock->autoflush(1);
            print "Sending $file_name\n$file_size bytes." ;
            print $sock "$file_name#:#" ; # send the file name.
            print $sock "$file_size\_" ; # send the size of the file t
+o server.

            open (FILE,$file) ;
            binmode(FILE) ;
            my $buffer = 0;
            while( sysread(FILE, $buffer , $bandwidth) ) {
                print $sock $buffer ;
                print "." ;
                sleep(1) ;
            }
            close (FILE);
            print $sock "__END";
            $sock->shutdown(1);   # close socket for writing
            $rc = 1;
        }
        else {
            print "Socket Error: \n";
        }
    }
    else {
        print "ERROR! Can't find or blank file $file: $!" ;
    }
    return($rc)
}

[download]

-----
Of all the things I've lost in my life, its my mind I miss the most.

Replies are listed 'Best First'.
Re: Transmit file over socket
by Roger (Parson) on Jan 20, 2004 at 07:17 UTC
    I think your server has a logic error. Consider your code:
       ...
   print "Got: $in\n";
   if ($in =~ /#:#$/) {
     ...
   } elsif ($in =~ /_$/) {
     ...
   } elsif ($in =~ /__END$/) {
     ...
   } else {
     ...
     # save file
   }

    [download]
    For the following input data: 
    Got: X-Call~4.xml#:#380_<?xml version="1.0"?>

    [download]
    Your logic will never get to the save file bit correctly, it will (almost) always fall inside the $in =~ /_$/ block and not saving the xml header correctly.


    You probably want something like this instead:
    if ($in =~ /__END$/) {
  # close previous file
} else {
  ($filename, $filesize) = $in =~ /^([^#]+)#:#(\d+)_/;

  if ($filename) {
     # create new file
  }

  # proceed with saving the file
}

    [download]

[reply]
[d/l]
[select]

      I will look into this - thank you. However, for the first file the filename and size are seperate (which is what I was looking for.. only for the second and following files that get processed are the filename and size and first line concatonated onto the same line. I suspect some kind buffering issue..?

      -----
      Of all the things I've lost in my life, its my mind I miss the most.
[reply]
        Could be. Thinking about the client/server program again, I would probably use the Net::TCP::Server module to implement the server. Much easier.
        use strict;
use warnings;
use Net::TCP::Server;

# create listener
my $lh = 'Net::TCP::Server'->new(8000) or die;

while (my $sh = $lh->accept) {
    defined(my $pid=fork) or die "fork: $!\n";
    if ($pid) {             # parent doesn't need client fh
        $sh->stopio;
        next;
    }
    
    # child doesn't need listener fh
    $lh->stopio;
    
    # do per-connection stuff here
    # read from the socket

    exit;
}

        [download]

[reply]
[d/l]
Re: Transmit file over socket
by halley (Prior) on Jan 20, 2004 at 15:27 UTC
    Pardon me for not reading two hundred lines of socket code before lunch, but the way the question was phrased, I think I might know the problem.

    Sockets provide streams of data. A message that took one call to send, may take three calls to read. Or vice versa. With TCP, you can't assume that all of your message will be available at the same time. You can only assume that it will all eventually arrive, in order, or the socket will close.

    Therefore, if the boundaries of data elements are important, then you need to put markers in the data stream itself, in such a way that you know if you have the whole data element, or even the whole marker, before you start using the data. If you don't have the whole marker, wait and read again, and then see if you have the whole marker.

    Again, sorry for not studying the whole code, but as someone else posted, there are some good modules that already do TCP packets for you. Another place to look is Chapter 12 of Advanced Perl Programming (by Srinivasan).

    --
    [ e d @ h a l l e y . c c ]

[reply]

Back to Seekers of Perl Wisdom