Beefy Boxes and Bandwidth Generously Provided by pair Networks
good chemistry is complicated,
and a little bit messy -LW
 
PerlMonks  

Hidden Newline in Result

by phamalda (Novice)
on Mar 05, 2017 at 18:39 UTC ( [id://1183704]=perlquestion: print w/replies, xml ) Need Help??

phamalda has asked for the wisdom of the Perl Monks concerning the following question:

I am working in a Solaris 10 environment and using Perl v5.8.4. I have script designed to remove NULL values from a file but the result is producing hidden newline characters in each line. I am using a hash to determine where to begin in deleting these tildes.

The original file looks something like this.

MEPMD01~20080519~03132016040900AM~~~~1782115850871~OK~E~-KWH~~000005~4 +9~03132016120500AM~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~0 +0~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~ +00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~~~~~~~~~~~~~~~~~~~~~ +~~~~10~0.0~10~0.0~10~0.0~10~0.0~10~0.0~10~0.0~10~0.0~10~0.0~10~0.0~10 +~0.0~10~0.0~10~0.0~10~0.0~

You can see the area with tildes and nothing between them. This code will remove those empty tildes. There is also a tilde at the end which needs to remain.

#!/usr/bin/perl use strict; # Force good coding practices use warnings; # Providing warning during execution use Switch; # Necessary for Perl Case statement # Define variables that will be modified within the foreach loop below + and then # accessed outside that loop. my $posn; my $cut; my $fieldtwelve; my $discard; my @field; # Set input and output files to variables. Script must have user defin +ed input # file. If second argument does not exist, default to 'results.txt' fi +le. my $infile = $ARGV[0]; my $outfile = $ARGV[1] || 'resultfile.txt'; # If no arguments, display usage format to user. unless (@ARGV) { die "Usage: IncrementalDSTCorrection.pl <inputfile> [<outputfile>] \ +n"; } # Open input file. Show user error if file not found. open IN, '<', $infile or die "$infile not found in current location"; # Open output file. Should not error but if it does... open OUT, '>', $outfile or die "Cannot open $outfile"; # Place file header from source file into $header variable and place # into output file via OUT file handle my $header = <IN>; print OUT $header; # Use arrays nested in a hash to store field values used later in # the @field array. # The first value will be the corrected interval count. # The second value will be the first field to begin removing NULL valu +es. # Notice that this value changed depending on the interval time and ti +me # of receipt. # The third field indicates how many NULL values will be removed from +that # record to correct the interval values. my %incrementalchange = ( '0000051205' => [36,62,24], '0000051210' => [36,60,24], '0000051215' => [36,58,24], '0000051220' => [36,56,24], '0000051225' => [36,54,24], '0000051230' => [36,52,24], '0000051235' => [36,50,24], '0000051240' => [36,48,24], '0000051245' => [36,46,24], '0000051250' => [36,44,24], '0000051255' => [36,42,24], '0000050100' => [36,40,24], '0000050105' => [36,38,24], '0000050110' => [36,36,24], '0000050115' => [36,34,24], '0000050120' => [36,32,24], '0000050125' => [36,30,24], '0000050130' => [36,28,24], '0000050135' => [36,26,24], '0000050140' => [36,24,24], '0000050145' => [36,22,24], '0000050150' => [36,20,24], '0000050155' => [36,18,24], '0000050200' => [36,16,24], '0000151215' => [12,30,8], '0000151230' => [12,28,8], '0000151245' => [12,26,8], '0000150100' => [12,24,8], '0000150115' => [12,22,8], '0000150130' => [12,20,8], '0000150145' => [12,18,8], '0000150200' => [12,16,8], '0000301230' => [6,22,4], '0000300100' => [6,20,4], '0000300130' => [6,18,4], '0000300200' => [6,16,4], '0000600100' => [3,18,2], '0000600200' => [3,16,2] ); my $count = 0; my $incount = 0; my $outcount = 0; my $totalcount = 0; # The foreach loop below will do the following: # 1. read in one record at a time in the foreach # 2. chomp any whitespace from that line # 3. Split the line into an array delimited by tildes # 4. Acquire the timestamp from each record # 5. Determine whether that timestamp was in AM or PM # 6. Combine the interval value and timestamp to be used later # 7. Begin processing each line to remove the NULL values where they e +xist foreach my $line (<IN>) { $count = $count + 1; chomp($line); my @field = split "~", $line; my $arraysize = scalar @field; my $intervalcount = $field[12]; my $dstdate = substr($field[13], 0, 3); if ( $arraysize > 15 and $dstdate eq '0313') { my $ampm = substr($field[13], 14, 2); # Ensure the current record is in the AM if ( $ampm eq 'AM' ) { if ($field[5] eq 'EPHONE CABINET') { print "EPHONE CABINET\n"; } if ($field[5] eq 'EPENDENT LIVING') { print "EPHONE CABINET\n"; } my $timestamp = substr($field[13], 8, 4); my $intervaltime = $field[11].$timestamp; # Evaluate field 11 in the @field array to determine the i +nterval # Open a Perl switch statement to handle any possible inte +rvals switch ($field[11]) { case ('000005') { # Use the $timestamp variable to determine the arr +ival time of the record and take appropriate # actions below. switch ($timestamp){ case ('1205'){ if ( $arraysize > 58 and $field[62] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; my $posn = $incrementalchange{$inter +valtime}[1]; # Identify first position to begin removing NULL valu +es my $cut = $incrementalchange{$inter +valtime}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. my $discard = join '',splice(@field,$p +osn,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. # print OUT join ('~', @field) . "\n"; print OUT join '~',@field; print OUT "\n"; $incount = $incount + 1; } # End if else { print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1210'){ if ( $arraysize > 56 and $field[60] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1215'){ if ( $arraysize > 54 and $field[58] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1220'){ if ( $arraysize > 52 and $field[56] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1225'){ if ( $arraysize > 50 and $field[54] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1230'){ if ( $arraysize > 48 and $field[52] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1235'){ if ( $arraysize > 46 and $field[50] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1240'){ if ( $arraysize > 44 and $field[48] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1245'){ if ( $arraysize > 42 and $field[46] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1250'){ if ( $arraysize > 40 and $field[44] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1255'){ if ( $arraysize > 38 and $field[42] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0100'){ if ( $arraysize > 36 and $field[40] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0105'){ if ( $arraysize > 34 and $field[38] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0110'){ if ( $arraysize > 32 and $field[36] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0115'){ if ( $arraysize > 30 and $field[34] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0120'){ if ( $arraysize > 28 and $field[32] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0125'){ if ( $arraysize > 26 and $field[30] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0130'){ if ( $arraysize > 24 and $field[28] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0135'){ if ( $arraysize > 22 and $field[26] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0140'){ if ( $arraysize > 20 and $field[24] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0145'){ if ( $arraysize > 18 and $field[22] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0150'){ if ( $arraysize > 16 and $field[20] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0155'){ if ( $arraysize > 14 and $field[18] eq '' +) { # Ensure NULL values exist where expected. If not do n +othing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0200'){ if ( $arraysize > 13 and $field[16] eq '' +) { # Ensure NULL values exist where expected. If not do +nothing. $fieldtwelve = $field[12] - 12; + # Place the value of $fieldtwelve into the $field[12] element of + the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case # If none of the times above are not met, the +record is not between 12AM and 2AM # Insert the outstanding record into the outpu +t file else { # print OUT join ('~', @field) . "~\n"; print OUT join '~',@field; $incount = $incount + 1; } } # End inner switch } # End Case case ('000015') { switch ($timestamp){ case ('1215'){ if ( $arraysize > 28 and $field[30] eq '' +) { # Ensure NULL values exist where expected. If not do +nothing. # print "$arraysize\n"; $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1230'){ if ( $arraysize > 26 and $field[28] eq '' +) { # Ensure NULL values exist where expected. If not do noth +ing. $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('1245'){ if ( $arraysize > 24 and $field[26] eq '' +) { # Ensure NULL values exist where expected. If not do + nothing. $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0100'){ if ( $arraysize > 22 and $field[24] eq '' +) { # Ensure NULL values exist where expected. If not do +nothing. $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0115'){ if ( $arraysize > 20 and $field[22] eq '' +) { # Ensure NULL values exist where expected. If not do +nothing. $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0130'){ if ( $arraysize > 18 and $field[20] eq '' +) { # Ensure NULL values exist where expected. If not do +nothing. $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; # print OUT join ('~', @field) . "~\n" +; print OUT join ('~', @field) . "\n"; $incount = $incount + 1; } } # End Case case ('0145'){ if ( $arraysize > 16 and $field[18] eq '' +) { # Ensure NULL values exist where expected. If not do +nothing. $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case case ('0200'){ if ( $arraysize > 14 and $field[16] eq '' +) { # Ensure NULL values exist where expected. If not do +nothing. $fieldtwelve = $field[12] - 4; + # Place the value of $fieldtwelve into the $field[12] element of +the array $field[12] = $fieldtwelve; $posn = $incrementalchange{$interval +time}[1]; # Identify first position to begin removing NULL values $cut = $incrementalchange{$interval +time}[2]; # Identify number of NULL values to remove # Combine (splice) the values above ba +ck into the proper element of the @field array. # Provide a warning to the user if som +ething other than a NULL value is removed. $discard = join '',splice(@field,$posn +,$cut); warn "Warn : $discard discarded" if ($ +discard); # Place the current element from the @ +field array into the output file. # Add a tilde at the end of that recor +d to ensure proper formatting. print OUT join ('~', @field) . "\n"; # print OUT join '~',@field; $incount = $incount + 1; } # End if else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End Case # If none of the times above are not met, the +record is not between 12AM and 2AM # Insert the outstanding record into the outpu +t file else { # print OUT join ('~', @field) . "~\n" +; print OUT join '~',@field; $incount = $incount + 1; } } # End inner switch } # End Case } # End outer switch } # End Top if # This else statement will insert into the output file all rec +ords that are in the PM else { # print OUT join ('~', @field) . "~\n"; print OUT join ('~', @field) . "\n"; $outcount = $outcount + 1; } } else { print OUT join ('~', @field) . "~\n"; # print OUT join '~', @field; $incount = $incount + 1; } } # End foreach $totalcount = $incount + $outcount + 1; print "Incount is $incount and Outcount is $outcount\n"; print "Total is $totalcount\n"; # Close input and output files close IN; close OUT;

This seems to work rather well when I more the resulting file as seen below.

MEPMD01~20080519~03132016121100AM~~~~1782115847997~OK~E~-KWH~~000005~4 +6~03122016081500PM~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~0 +0~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~ +00~0.0~00~ ~.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~ +0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00 +~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~00~0.0~
However, if I vi the file, there seems to be a dangling tilde following the newline character (^M) and I can't figure out where it is coming from.

MEPMD01~20080519~03132016120400AM~1539065607~42~~1782115849352~OK~E~kV +ARhtd~~000005~48~03122016080500PM~00~0.0191~00~0.0194~00~0.0195~00~0. +0195~00~0.0192~00~0.0194~00~0.0197~00~0.0197~00~0.0192~00~0.0194~00~0 +.0194~00~0 .0192~00~0.0194~00~0.0195~00~0.0194~00~0.0197~00~0.0197~00~0.0195~00~0 +.0195~00~0.0195~00~0.0195~00~0.0195~00~0.0201~00~0.02~00~0.02~00~0.02 +~00~0.0201~00~0.02~00~0.02~00~0.0201~00~0.0203~00~0.02~00~0.0203~00~0 +.0201~00~0 .0203~00~0.0206~00~0.0207~00~0.0204~00~0.0206~00~0.0207~00~0.0207~00~0 +.0206~00~0.0209~00~0.0207~00~0.0207~00~0.0207~00~0.0209~00~0.0207~^M~

I really need to get rid of that pesky tilde following the newline character. Any help would be appreciated.

Replies are listed 'Best First'.
Re: Hidden Newline in Result
by dave_the_m (Monsignor) on Mar 05, 2017 at 20:00 UTC
    It's not directly related to your issue, but note that your code is needlessly over-complex, and could be simplified enormously. Every case code block is virtually identical apart from (AFAIKT) differences in the constants for the comparisons and minor differences in what is output. This could be handled as a simple hash of hashes of arrays. In outline:
    my %actions = ( '000005' => { '1205' => [ 58, 62, ... ], '1210' => [ 56, 60, ... ], ... }, '000015' => { '1215' => [ 28, 30, ... ], '1230' => [ 26, 28, ... ], ... }, ); ... my $action = $actions[$field[11]; if ($action) { my $action2 = $action->{$timestamp}; if ($action2) { my ($maxarray, $field_ix, $whatever_else) = @$action2; if ($arraysize > $maxarray and $field[$field_ix] eq '') { .... } else { .... } } else { .... } } else { .... }
    In any case, don't rely on Switch: it's buggy and was removed from the perl core 6 years ago.

    Dave.

Re: Hidden Newline in Result
by poj (Abbot) on Mar 05, 2017 at 19:05 UTC

    If the file is from windows try changing the chomp to this

    #chomp($line); $line =~ s/[\r\n]+$//g;
    poj
Re: Hidden Newline in Result
by Corion (Patriarch) on Mar 05, 2017 at 19:05 UTC

    You are using the Switch module, which can cause weird situations due to its code rewriting nature.

    As a first step, have you tried rewriting your code without use of the Switch module?

    As a second step, remove parts of your code until the problem disappears. Most likely, the part you removed contains a part of the problem. You have posted far too much code and far too little input data for us to properly attempt to reproduce your problem. See SSCCE for what helps us help you better.

    Note that you are using chomp:

    chomp($line);

    This will not remove any ^M characters from the end of $line.

    Most likely, you want to remove all whitespace from the end of the line instead, or all "newline-ish characters":

    $line =~ s![\r\n]+$!!; # just newlines $line =~ s!\s+$!!; # all kinds of whitespace

      Thank you so much for the quick feedback. I really appreciate it. I actually think I've solved it. I removed that tilde from the end of big If/Then/Else statement.

      else { print OUT join ('~', @field) . "\n"; # print OUT join '~', @field; $incount = $incount + 1; }

      It appears that this has produced the desired result. I'll know more tomorrow when we can consume the resulting file. Thanks again for everything.

Re: Hidden Newline in Result
by poj (Abbot) on Mar 05, 2017 at 21:14 UTC

    I suggest you check this bit of code with regard to substr

    my $dstdate = substr($field[13], 0, 3); if ( $arraysize > 15 and $dstdate eq '0313') {

    $dstdate will be 3 characters in length (at most) so it can never equal 0313

    poj

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1183704]
Approved by Athanasius
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others drinking their drinks and smoking their pipes about the Monastery: (4)
As of 2024-03-19 07:40 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found