Beefy Boxes and Bandwidth Generously Provided by pair Networks
go ahead... be a heretic
 
PerlMonks  

Transmit file over socket

by AcidHawk (Vicar)
on Jan 20, 2004 at 06:33 UTC ( #322519=perlquestion: print w/replies, xml ) Need Help??

AcidHawk has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

I am trying to read a dir for small files on one machine and if a file exists transmit it over a socket I have created to another machine. These files are typically small xml files but coule also be small zip files. (I still need to put in checking for large files to ignore them.) The problem I have is that the first files goes over and gets saved successfully BUT the next file is not correct. The filename and size and first line of the next file is sent over in one step. So my output looks as follows

Got: X-Call~4.xml#:#380_<?xml version="1.0"?>
instead of
Got: X-Call~4.xml#:# Got:380_ Got:<?xml version="1.0"?>

So this is what I have...

Server...

#!/usr/bin/perl use strict; use warnings; use IO::Socket; use IO::Select; use Time::Local; use File::Basename; ###################################################################### +################## #xml Variables my ($xinput_path, $xmlout, $debug, $port, $logfile, $new_file, $rules) +; my ($hb, $hb_prev, $now, $hb_email, $hb_duration, $buffer, $max_client +s, $slm); my ($checkdir_thresh, $checkdir_duration, $checkdir_prev, $manager); #Socket Variables my ($remote, $sock, @client_list, $sel, $new_remote); #Misc Variables my ($VERSION); ###################################################################### +################## my $rc = 0; $VERSION = "1.0.0"; my $timeout = 5; my $save_dir = './files' ; unless (&Create_Socket()) { print "Problem with Creating a Socket: $!\n"; exit(99); } my ($filesave, $filesize, $filename, $data_content); while(1) { my @client_list = $sel->can_read(1); foreach $remote(@client_list) { if ($remote == $sock) { $new_remote = $sock->accept; $sel->add($new_remote) if ($new_remote); } else { my $in; if (sysread($remote, $in, 1024)) { if (defined $remote) { print "Got: $in\n"; if ($in =~ /#:#$/) { $filename = $in; $filename =~ s/#:#$// ; print "Filename = $filename\n"; } elsif ($in =~ /_$/) { $filesize = $in; $filesize =~ s/_$// ; print "Filename = $filename\n"; print "Filesize = $filesize\n"; } elsif ($in =~ /__END$/) { close (FILENEW) ; print "...\n\n" ; $sel->remove($remote); close($remote); ($filesave, $filesize, $filename, $data_conten +t) = 0; } else { print "About to save file\n"; unless ($filesave) { $filesave = "$save_dir/$filename" ; print "FileSave - $filesave\n"; if (-e $filesave) { unlink ($filesave) ; } print "Saving: $filesave ($filesize bytes) +\n" ; } open (FILENEW,">>$filesave") ; binmode(FILENEW) ; print FILENEW $in ; } } else { print "DEBUG - \$remote Client NOT defined\n +"; } } else { print "WARNING - X-InputServer Could not Read data + from a connected socket: @_\n"; $sel->remove($remote); close($remote); } } } } exit($rc); ###################################################################### +################## ## ## SUB ROUTINES ## ###################################################################### +################## #Create a Socket to listen on sub Create_Socket { $SIG{CHLD} = 'IGNORE'; $sock = IO::Socket::INET->new( LocalPort => 6123, Proto => 'tcp', Reuse => 1, Listen => 100 ) or return(0); $sel = IO::Select->new($sock); print "INFORMATION - Socket Created ... Successful\n"; print "INFORMATION - Awaiting messages on: 6123\n"; return(1); }
Client...
#!/usr/bin/perl use strict; use warnings; use IO::Socket; #Socket Handeling Functions use File::Basename; #To handle the path and filename functions ###################################################################### +################## #program version my $VERSION="4.0.0"; #Setup Vars my ($xbridge_path, $sock); my $rc = 0; my $port = "6123"; my $srv = "192.x.x.x"; ###################################################################### +################## if (&Setup_Log()){ while(1) { my @list = (); if (opendir DIR, "./files") { while (my $file = readdir DIR) { next if $file =~/^\./; push @list, $file; } } if ($list[$#list]) { print "about to process ./files/$list[$#list]\n"; if (&Transmit("./files/$list[0]")) { print "Moved ./files/$list[0]\n"; unlink("./files/$list[0]"); } } else { print "Nothing to process yet\n"; sleep(1); } } } else { print "X-Bridge Could Not Perform the Setup_Log: $!"; $rc = 1; } exit ($rc); ###################################################################### +################## ## ## SUB ROUTINES ## ###################################################################### +################## #Brief Setup_Log to initiate the LOG file sub Setup_Log { my $rc = 0; # Open Config File and Read Values into Hash my $path = "$ENV{'MyConfigDir'}/XInput"; if ($path =~ /:/) { if ($xbridge_path = Win32::GetShortPathName($path)) { $xbridge_path =~s /\\/\//g; print "path = $path\n"; $rc = 1; } else { print "$0 - Path Problems in Setup for $path: $^E\n"; } } else { print "path = $path\n"; $rc = 1; } return($rc); } sub Transmit { my $rc = 0; my $bandwidth = 1024*5 ; # 5Kb/s my $file = shift; if (-s $file) { my $file_size = -s $file ; my ($name, $path, $type) = fileparse($file, '\..+'); my $file_name = "$name$type"; print "Filename = $file_name\n"; if ($sock = new IO::Socket::INET ( PeerAddr => $srv, PeerPort => $port, Proto => 'tcp', )) { $sock->autoflush(1); print "Sending $file_name\n$file_size bytes." ; print $sock "$file_name#:#" ; # send the file name. print $sock "$file_size\_" ; # send the size of the file t +o server. open (FILE,$file) ; binmode(FILE) ; my $buffer = 0; while( sysread(FILE, $buffer , $bandwidth) ) { print $sock $buffer ; print "." ; sleep(1) ; } close (FILE); print $sock "__END"; $sock->shutdown(1); # close socket for writing $rc = 1; } else { print "Socket Error: \n"; } } else { print "ERROR! Can't find or blank file $file: $!" ; } return($rc) }

-----
Of all the things I've lost in my life, its my mind I miss the most.

Replies are listed 'Best First'.
Re: Transmit file over socket
by Roger (Parson) on Jan 20, 2004 at 07:17 UTC
    I think your server has a logic error. Consider your code:
    ... print "Got: $in\n"; if ($in =~ /#:#$/) { ... } elsif ($in =~ /_$/) { ... } elsif ($in =~ /__END$/) { ... } else { ... # save file }
    For the following input data:
    Got: X-Call~4.xml#:#380_<?xml version="1.0"?>
    Your logic will never get to the save file bit correctly, it will (almost) always fall inside the $in =~ /_$/ block and not saving the xml header correctly.


    You probably want something like this instead:
    if ($in =~ /__END$/) { # close previous file } else { ($filename, $filesize) = $in =~ /^([^#]+)#:#(\d+)_/; if ($filename) { # create new file } # proceed with saving the file }

      I will look into this - thank you. However, for the first file the filename and size are seperate (which is what I was looking for.. only for the second and following files that get processed are the filename and size and first line concatonated onto the same line. I suspect some kind buffering issue..?

      -----
      Of all the things I've lost in my life, its my mind I miss the most.
        Could be. Thinking about the client/server program again, I would probably use the Net::TCP::Server module to implement the server. Much easier.
        use strict; use warnings; use Net::TCP::Server; # create listener my $lh = 'Net::TCP::Server'->new(8000) or die; while (my $sh = $lh->accept) { defined(my $pid=fork) or die "fork: $!\n"; if ($pid) { # parent doesn't need client fh $sh->stopio; next; } # child doesn't need listener fh $lh->stopio; # do per-connection stuff here # read from the socket exit; }

Re: Transmit file over socket
by halley (Prior) on Jan 20, 2004 at 15:27 UTC
    Pardon me for not reading two hundred lines of socket code before lunch, but the way the question was phrased, I think I might know the problem.

    Sockets provide streams of data. A message that took one call to send, may take three calls to read. Or vice versa. With TCP, you can't assume that all of your message will be available at the same time. You can only assume that it will all eventually arrive, in order, or the socket will close.

    Therefore, if the boundaries of data elements are important, then you need to put markers in the data stream itself, in such a way that you know if you have the whole data element, or even the whole marker, before you start using the data. If you don't have the whole marker, wait and read again, and then see if you have the whole marker.

    Again, sorry for not studying the whole code, but as someone else posted, there are some good modules that already do TCP packets for you. Another place to look is Chapter 12 of Advanced Perl Programming (by Srinivasan).

    --
    [ e d @ h a l l e y . c c ]

Log In?
Username:
Password:

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://322519]
Approved by markmoon
Front-paged by bart
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others rifling through the Monastery: (5)
As of 2021-03-01 16:51 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    My favorite kind of desktop background is:











    Results (13 votes). Check out past polls.

    Notices?