sub split_logfile(){
     print "In sub split_logfile\n";
}
sub gen_stats(){
     print "In sub gen_stats\n";
}
sub compress_logs(){
     print "In sub compress_logs\n";
}
sub clean_up(){
     print "In sub clean_up\n";
}
[download]

Right. I put

print("DONE\n");
[download]

At the end of split_logfile() and

print("Starting gen_stats\n");
[download]

at the beginning of gen_stats(). It printed the messages out in correct order:

DONE
Starting gen_stats
...
[download]

So it appears that the functions are getting called in the correct order.

The reason this can happen is that the I/O is processed, and then left for the OS to pick up and display. In the mean time, the computer sees no need to just wait around twiddling it thumbs, so it goes on and executes the next line of code. It may very well be able to execute several of lines of code (depending on their complexity/operations) in the time it takes to do a single print statement. But if one of those lines in turn sends something to STDERR, then that could end up getting shunted to the front of the I/O list, because STDERR is given higher priority than STDOUT on most platforms.

So if you have the lines:

print "Hello there\n";
print STDERR "My name is bob\n";
[download]

There is no guarantee which line will be printed first. Usually "hello there" will make it to the screen first, but there is no guarantee of that (unless perl does something behind the scenes that I'm not aware of).

And screen I/O is faster than file I/O, so if you've also got output to a file that you're tail -f'ing, that can appear all out of order, too.

Of course, this could all be completely unrelated to your problem, but I hope it helps (or was at least mildly educational).

Some people drink from the fountain of knowledge, others just gargle.

Seriously, perhaps you simply didn't finish processing the data before moving on to the next function? These functions are executed sequentially.

AgentM Systems nor Nasca Enterprises nor Bone::Easy nor Macperl is responsible for the comments made by AgentM. Remember, you can build any logical system with NOR.

So if your program is small, just post it completely, otherwise post a bit more of your program and of the subs.
Or, even better, post a url where it can be seen completely.

sub split_logfile {
    # As adapted from the split-logile perl script from the Apache gro
+up
    my($vhost);
    my($log_line);
    my(%is_open);

    open(LOG, "<$access_log")
        or die("Cannot open $access_log for processing: $!");

    while($log_line = <LOG>) {
        ($vhost) = split(/\s/, $log_line);
        $vhost = lc($vhost);

        if (! $is_open{$vhost}) {
            open $vhost, ">>$work_dir/${vhost}.log"
                or die ("Cannot open ${vhost}.log: $!");
            $is_open{$vhost} = 1;

        }

    $log_line =~ s/^\S*\s+//;
    printf($vhost "%s", $log_line);
    }

    print("DONE\n");
}
[download]

sub gen_stats {
    my($log);
    my($domain);

    # Generate stats for the entire server (server wide)
    system($webalizer, "-q", "-D", $work_dir."/dns_cache.db", "-N", "2
+0", "-n", $server_hostname,
            "-o", $server_log_dir, $access_log);

    opendir(WORKDIR, $work_dir) or die "Could not open $work_dir: $!\n
+";

    while (defined($log = readdir(WORKDIR))) {
        # If the file isn't . or .. then strip off the .log portion to
+ get just the host
        # + domain. Run webalizer using log file.
        if ($log !~ /^\.\.?$/ and $log !~ /dns_cache\.db/) {
            ($domain) = split(/\.log/, $log);

            # IF the web stats directory doesn't exist, don't bother r
+unning stats for 
            # that particular web.
            if ( -e "$stats_dir/$domain" ) {
                system($webalizer, "-q", "-D", $work_dir."/dns_cache.d
+b", "-N", "20",
                    "-n", $domain, "-o", $stats_dir."/".$domain, $work
+_dir."/".$log);
            } else { print "Stats directory for $domain doesn't exist.
+\n"; }
        }
    }
    closedir WORKDIR;
}
[download]

Thanks again.

Brendon

close $vhost or warn "Failure closing $vhost: $!\n";
[download]

        - tye (but my friends call me "Tye")

I don't know exactly what your code if doing, but is gen_stats is trying to read a file split_logfile is creating, you might be running into a buffering problem.

Try putting this line at the beginning of split_logfile:

local $| = 1;

This will turn autoflush on, which means Perl will output to a file immediately after a print, rathering than saving outputs up and doing them all at once.

That would only autoflush STDOUT, which probably isn't the problem in this case.

        - tye (but my friends call me "Tye")

What platform are you on?

On Windows when dealing with shared filesystems I have been burned enough that I routinely make sure that I have waited a substantial time (eg one second) between when I closed a file and when I do anything which depends on my write being visible to the world. Why this should be necessary I do not know, but then again I likewise don't bother to remember the exact combination of versions of Perl and Windows in which renaming a file from FOO to foo will delete the file because it does not realize that the target is the original file. (So delete the existing, then go to ren...oops.)

I merely know that I was burned and the absolutely safe fixes were to sleep in the one case or in the other to rename into another directory and then rename back. (Contrary to Microsoft's documentation on the bug, renaming through another name in the same directory was not safe.)

Anyways it is possible that you are hitting bugs...

I must have overlooked something. I did some further testing, and it appears that if I don't close the files that I've opened then the data does not get fully written to them. I should have known this... but in the original script from the Apache source it didn't do this - probably because it only had one routine to do and then the script ended, closing all the files automatically.

Thanks to all for the help!