Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Origin of 'md5sum' and 'sh' logs

by drw (Novice)
on Feb 26, 2019 at 00:02 UTC ( [id://1230539]=perlquestion: print w/replies, xml ) Need Help??

drw has asked for the wisdom of the Perl Monks concerning the following question:

Edit: I have identified the error (Thanks bliako). I was not using the explicit $exception->{exception} value and this was causing the confusion/inconsistencies in the code. Please excuse me for the confusing indentations and format on this post. I tried to edit down the original code to something more readable, but failed miserably at it. Thank you everyone for the advice as well, it is very appreciated.

Couldn't figure out a way to search this problem, so I apologize if it has been answered before. Additionally, I am very new to perl, excuse me if i overlook something obvious.

I am working on creating an email alert for a data load that goes through every day at my company. There are a number of modules that do essentially the same thing a few different ways for the different sets of data. What I was tasked to do is create alerts whenever the data is not loading properly, specifically when the file does not exist. This was pretty simple to start, but I've hit a roadblock for a few of the modules.

Currently I have a subroutine that reads:

if (-e $filepath) { # the file exists if (-M $filepath > $maxage_days) { my $exception = { table => $tablename, exception => 'outofdate', data => { file => $filepath, age => sprintf("%.2f", (-M $filepath)*24) . ' hou +rs', still_copying => 0 } return ($exception); }; else { my $exception = { table => $tablename, exception => 'nonexistent', data => { file => $filepath, age => 0, nonexistent => 1 } }; return ($exception); }

The $exceptions are returned to one of the main modules (whichever is currently being used) and proceeds to run down its every day instructions.

I have implemented another if statement to check if the returned $exception is "nonexistent":

if ($exception eq 'outofdate' or $exception eq 'stillcopying') { $biuw->{log}->warn("$FEED_PATH . $FEED_FILE_BASE_NAME is out o +f date. Loading stopped.\n"); return $exception } if ($exception eq 'nonexistent') { $biuw->{log}->warn("$FEED_PATH . $FEED_FILE_BASE_NAME does not + exist. Loading stopped.\n"); return $exception; }
The majority of the results are exactly what I need. The script fails, a log stating that the $filepath does not exist, and an email is sent out to our alerts queue (this is done in the main module that organizes all the others.) What I am getting with two of them, is instead of proceeding the normal way, I get these logs:
sh: /home/ftp01.mbira.com/reports/futu.data: No such file or directory md5sum: /home/ftp01.mbira.com/reports/futu.data: No such file or direc +tory

The script continues to attempt to load the data, but hits a number of issues as the file path does not exist. I am not sure why two of the modules are creating these logs instead what is instructed, but the other modules will simple generate the error and email and then end.

Any thoughts on where to look for trouble shooting this error, or maybe some kind of explanation of why the -e command is generating these md5sum and sh logs?

Replies are listed 'Best First'.
Re: Origin of 'md5sum' and 'sh' logs
by bliako (Monsignor) on Feb 26, 2019 at 10:53 UTC

    This can be irrelevant to your current problems but can save you on future problems: -M $filepath can return a negative age too! :    -M  Script start time minus file modification time, in days. (see -X).

    Secondly, you are creating "exception" being some kind of hash with a key also called "exception" and then in your last code you compare $exception as a string. I would doublecheck that this $exception is indeed $exception->{'exception'} and not the original hash exception.

    Thirdly and most importantly, the first chunk of code you showed has an if-statement which the first branch returns nothing. So your "age" exceptions are not returned. The second branch does return an exception HASH though.

    Perhaps, place my $exception; after if( -e $filepath), then go through all your exception if-checks, remove all other my $exception... and finally return $exception only if it is defined. To achieve a state of lift-off you need to insert two more closing curly brackets.

      So, you hit the nail on the head with the "exception" hash on point number two. It's weird that this hasn't come up with the other modules.

      In the interest of making everything clear for future users, I am going to make sure that any reference to the $exception hash and the $exception->{exception} scalar value is made explicit. It would seem that the original programmer only had to worry about the $exception merely being defined, and in my ignorance, ignored the value of the $exception->{exception} value, even though I was defining it in my return.

      I've only been working with perl for the past few months, sop if my vocabulary is off, please excuse me. Thank you for bringing that to my attention though, this has solved my problem.

Re: Origin of 'md5sum' and 'sh' logs
by soonix (Canon) on Feb 26, 2019 at 06:43 UTC
    sh: /home/ftp01.mbira.com/reports/futu.data: No such file or directory md5sum: /home/ftp01.mbira.com/reports/futu.data: No such file or direc +tory
    From intuition, I'd say these come from a shell script, not from your Perl script. Or perhaps (even more likely) a CMD script. Which OS are you working with?
      I am running this in Mac OS 10.13 terminal.
Re: Origin of 'md5sum' and 'sh' logs
by hippo (Bishop) on Feb 26, 2019 at 09:05 UTC
    Any thoughts on where to look for trouble shooting this error, or maybe some kind of explanation of why the -e command is generating these md5sum and sh logs?

    It isn't. Whatever is generating that output is not in the code you have shown. You should reduce your entire code to the smallest possible script which does generate those errors (an SSCCE) and provide that if you want more help to debug it. Without that, we can only guess (although my money is on one of either a race condition or more likely the $filepath in your subroutine isn't the path you think it is).

    In the meantime, check out the Basic debugging checklist.

Re: Origin of 'md5sum' and 'sh' logs
by Lotus1 (Vicar) on Feb 26, 2019 at 17:30 UTC
    if (-e $filepath) { # the file exists if (-M $filepath > $maxage_days) { my $exception = { table => $tablename, exception => 'outofdate', data => { file => $filepath, age => sprintf("%.2f", (-M $filepath)*24) . ' hou +rs', still_copying => 0 } };

    This won't compile since you don't have the same number of opening and closing curly brackets, '{' and '}'. Your indentation is confusing folks also. One suggestion is to use a programming text editor that will help you keep track of opening and closing brackets and do syntax highlighting.

Re: Origin of 'md5sum' and 'sh' logs
by Marshall (Canon) on Feb 26, 2019 at 01:14 UTC
    I am not sure what's going on here. But do you have use strict; use warnings; at the top of your script?

    if (-e $filepath) { my $exception... } else { my $exception... }
    shouldn't compile. In general a "my" variable cannot be declared within a conditional statement.
    EDIT: yes this will compile, but $exception cannot be used outside of its lexical scope.

    $exception is reference to a hash:

    #!/usr/bin/perl use strict; use warnings; my $exception = {table => 'tablename', exception => 'outofdate'}; my $exception2 = {table => 'tablename2', exception => 'allOk'}; foreach my $error ($exception, $exception2 ) { print "$error->{exception}\n"; } __END__ outofdate allOk
      In general a "my" variable cannot be declared within a conditional statement.
      Of course it can. Why should this be a problem?

      You can declare a "my" variable in any lexical scope - the branches of an if-statement are just one example:

      use strict; my $whatever; if($whatever) { my $something = "here"; } else { my $something = "there"; } print "look I compiled\n";
        can declare a "my" variable in any lexical scope. Well, ok but you can't use it outside of that scope. The OP's indenting is confusing to me. Your code works until you try to use $something outside of lexical scope.
        #!/usr/bin/perl use strict; use warnings; my $whatever; if($whatever) { my $something = "here"; } else { my $something = "there"; } print "look I compiled $something\n"; __END__ Global symbol "$something" requires explicit package name (did you for +get to declare "my $something"?) at Monks\Badcode.pl line 14. Execution of Monks\Badcode.pl aborted due to compilation errors. Process completed with exit code 255

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1230539]
Approved by Athanasius
Front-paged by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others perusing the Monastery: (5)
As of 2024-04-24 04:02 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found