Hello Monks
The script I am writing will be implemented on a windows device. I'm not sure if the reason why it fails on windows but works in mac is due to incompatibility of commands
Overview of the script. I have A LOT of fancy binary 'chp' files that need to be converted to txt files. The only way to convert this file is through a proprietary program
cont. The script takes an array of the directories of the files, chops them up into smaller arrays of 20. For each 20 files it forks into 20 child processes which sends the command for text conversion into system. Then it parses through the file and gets the data I want
#!/usr/bin/perl
use strict;
use File::Find;
use Parallel::ForkManager;
my @toParse; # array of chip files
foreach(@toParse){
my $encryptKey = $_; #Encrypted for patients
my @name = split /\//, $fileManifest{$encryptKey}; #hash contains
+the directories
my $id = pop(@name); # @name is an array of the directory, last el
+ement is the sample ID
my $perlOut = join "\/",@name; # directory for perl commands
my @safeID = split / /, $id;
my $safeDirect = $perlOut."/".$safeID[0]; # hipaa compliant
next if($safeID[0 ] =~ /cgo/ig);
#-----------------------------------------------------------------
# Prepare the commands for conversion
+
#-----------------------------------------------------------------
my $cychpCommand = $fileManifest{$encryptKey}; #directory of the c
+ychp file
$cychpCommand =~ s/\//\\/g; # Format the directory for windows and
+ for the affy program
$cychpCommand = "\"".$cychpCommand."\"";
my $cychpTxtOut = join "\\", @name; #CYCHP TXT OUTPUT DIRECTORY
+
$cychpTxtOut ="\"".$cychpTxtOut."\"";
+
my $affyCommand = "apt-chp-to-txt -o ".$cychpTxtOut." ".$cychpComm
+and;
unless(exists($cychpLog{$encryptKey})) {
my $rmCom = $perlOut."/apt-chp-to-txt.log";
push @toConvert1, $affyCommand; # To send commands for conversion
push @toConvert2, $rmCom; # To remove the log files
$cychpLog{$encryptKey}++;
$timestamp = localtime(time);
open LOG, ">>".$logFileName;
print LOG "$encryptKey\t$safeID[0]\t$safeDirect\t-\t-\t$timestamp
+\n";
close(LOG);
}
}
#-----------------------------------------------------------------
# Begin forking
#-----------------------------------------------------------------
my $maxProcs = scalar(@toConvert1);
my $pm = new Parallel::ForkManager($maxProcs);
foreach(@toConvert1){
my $pid = $pm-> start and next; # FORK Begin
system($_); # CONVERSION COMMAND
$pm->finish; # End of FORK
}
$pm->wait_all_children; # Blocks until all children are finished
foreach(@toConvert2){
unlink $_; #Delete log files made by the conversion program
}
foreach(@toParse){
my $txtFile = $_.".txt";
open IN, $txtFile or die "Cannot open [$txtFile]\n";
while(<IN>){
parsing happens here
}
}
So my question is as follows,
Is my code converting the files, and then waiting until they are all finished? If not then why not? How can I accomplish this on a windows machine. I cannot convert all files at once because the resulting files are too large. I have to convert a few, parse them, and then unlink them later. The error I'm getting is that the proprietary program cannot open the log file, I presume it's deleted because I unlink them right after the conversion takes place. The only way I think this error is occurring is that the script moves on without waiting for the forking to finish