Perhaps rather than reinventing wheels use AnyEvent and EV to setup a timer event to be posted every second.

$ perl -MTime::HiRes=time -MAnyEvent -E '$quit = AnyEvent->condvar;$w 
+= AnyEvent->timer( after => 1, interval => 1, cb => sub { state $ctr 
+= 1; say "@{[Time::HiRes::time()]}: FOOP! $ctr"; $quit->send if 10 ==
+ $ctr++;  } );say qq{Starting loop};$quit->recv;'
Starting loop
1574791289.26471: FOOP! 1
1574791290.26702: FOOP! 2
1574791291.26394: FOOP! 3
1574791292.26511: FOOP! 4
1574791293.26735: FOOP! 5
1574791294.26826: FOOP! 6
1574791295.26665: FOOP! 7
1574791296.265: FOOP! 8
1574791297.26659: FOOP! 9
1574791298.26699: FOOP! 10
$
[download]

Thanks, but using low-level commands is faster. For us speed (and thus accuracy) is very important.

I have tested your code on my computer and the average difference from a second is +0.00006222222222222222, which again is more by two decimal places than the difference of the negative_timer (which is -0.000000676723442).

EV is an interface to the lower-level libev, and AFAIK it's fairly performant. Its "periodic" function claims to work with wall clock time, so you shouldn't have any drift relative to the system clock. That the system clock can drift relative to real time, and that it can be adjusted in jumps, are whole different issues. I'm not sure if the default ntpd supports smooth adjustments to the system clock (Update: apparently, it does), butand I believe chrony does. (I have several systems with a GPS receiver that supports the PPS signal, and I feed that into chronyd to provide a system time that should be quite accurate. I've published some info on that here.)

use warnings;
use strict;
use Math::BigFloat;
use Time::HiRes qw/gettimeofday/;
use EV;

my $w1 = EV::signal( 'INT', sub { EV::break } );

my ($cnt, $avg, $prev) = (0);
my $w2 = EV::periodic( 0, 1, 0, sub {
    my $cur = Math::BigFloat->new( sprintf("%d.%06d",gettimeofday) );
    if ( defined $prev ) {
        my $delta = $cur - $prev;
        $avg = defined $avg ? ( $avg + $delta ) / 2.0 : $delta->copy;
        $cnt++;
    }
    $prev = $cur->copy;
} );

EV::run;

print "\nAverage over $cnt deltas: $avg\n";
[download]

Sample output from different runs on my system:

Average over 67 deltas: 1.000005597916695584862263807609389365538
Average over 76 deltas: 1.00012161307435464691799175142220832736
Average over 137 deltas: 0.9997675784785176200109895891945901313775
Average over 442 deltas: 0.9999437250604076389815434390305220372265
[download]

Why is sub micro-second time important and if it is why are you using Perl which implies a non-real time OS? Even using dedicated hardware for time critical applications with that degree of time precision can be challenging so the implication of your post is that you have made a major challenge for yourself!

Agreed. Even if you, say, used an Arduino with an external crystal for triggering interrupts you will have trouble hitting the exact timing. That is, unless you spend quite a lot on a very precise, temperature controlled and calibrated timing source.

If you want really precise, reproducable timing, you probably have to put your time critical stuff in an FPGA (or even better: ASIC) and hook that up to a very, very precise timing source, both for chip clock cycles and the external triggering interrupt. There are chip scale atomic clocks and things like that available for that purpose.

To make the data logging even more time stable, you also need to make sure your time critical code path is always the same, otherwise you introduce clock cycle count jitter. That means no conditionals from the start of the interrupt to having gathered all time critical data values in memory.

...basically, the more precision you want, the more specialized engineering you have to do. This can get expensive really, really fast. And there will always be compromises involved. It's the old mantra of "good/fast/cheap - pick any two of those".

perl -e 'use Crypt::Digest::SHA256 qw[sha256_hex]; print substr(sha256_hex("the Answer To Life, The Universe And Everything"), 6, 2), "\n";'

For finer resolution you could try select rbits,wbits,ebits,timeout, though I don't think this is what you are really after.

select(undef,undef,undef, $start - time());
#...4962.38324
#...4963.39886
#...4964.41448
[download]

If you just plug in your calculated difference that gets closer. But again, I don't think thats really what you are after.

select(undef,undef,undef, 0.9993);
#output
#....5554.40476
#....5555.40476
#....5556.40476
[download]

I was wondering if there was a solution that could be derived from using the complements of the numbers, as I have recently been looking at these. I got as close as 0.00129 difference from a second, on the first iteration. Whether its truly to spec, or could be refined, I am not sure, though it looks like something might be there.

The interval must be as precise as possible.

I think that this is causing some confusion in this thread, and we should take a moment to define the problem better. See e.g. Accuracy and precision - as several people have commented, if you want precise measurement of time, then really the only way to go about it is a real-time OS and/or other custom circuitry connected to an atomic clock, high-quality GPS receiver, etc. However:

We have monitoring scripts that need to be run every second. The scripts execution takes longer than a second, but that is not a problem. ... I know that the preciseness depends on CPU, RAM, CMOS etc. I don’t care about these.

This, along with the code loop you showed, leads me to believe that precision isn't the central point of your question, it seems to me more like you're worried about a systematic and cumulative error, which would mean you're actually more worried about trueness.

In other words: You want something like for (1..100) { sleep( 1 + $correction_factor ) } to take as close to 100 wall clock seconds as possible. And at the same time, perhaps it's not so important to you if each delay has e.g. ±0.1 seconds precision, as long as they even out, in other words, it's ok if 10 of the delays take 1.1 seconds, as long as 10 other delays take 0.9 seconds.

Is this interpretation of your problem correct? In that case, IMHO sleep is not the right tool for the job, as I showed you'll need something that works with the computer's wall clock time.

And keeping the computer's wall clock time in sync with real-world time is a whole different can of worms! But in general that's what NTP is for, and if you're having trouble with that, as you seem to indicate, then perhaps it's best to look at that as a separate topic. (Update: Actually, on a second reading, apparently it's the corrections made to the system clock by NTP that seem to be giving you the trouble? That seems strange, if you're really running chronyd and no other NTP client on this machine. But in any case, everything above still stands.)

Update: I'm putting this in my top level reply because IMHO it is the best answer to your question: I only realized further down in this thread that clock_nanosleep from Time::HiRes is probably what you're looking for.

haukex, you’re the closest to understanding my problem and possibly to give me a solution!

Well, on the chronyd stuff: I am running chronyd service, but not running ntpd (which is not even installed on particular system), see bellow.

$ systemctl status chronyd
● chronyd.service - NTP client/server
   Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; v
+endor preset: enabled)
   Active: active (running) since Wed 2019-10-23 13:38:15 CEST; 1 mont
+hs 4 days ago
     Docs: man:chronyd(8)
           man:chrony.conf(5)
 Main PID: 8102 (chronyd)
   CGroup: /system.slice/chronyd.service
           └─8102 /usr/sbin/chronyd

Warning: Journal has been rotated since unit was started. Log output i
+s incomplete or unavailable.

$ systemctl status ntpd
Unit ntpd.service could not be found.
[download]

I still need to delve into the chronyd docs.

Is this interpretation of your problem correct?

Well, kind of. We’d like to monitor some stuff each and every second. In the perfect world, this would mean that each and every real second a script would run, get data and insert in into the DB. As we don’t want/need to use atomic clock (etc), therefore by the real second we mean a second as precice accurate as possible. … In the end, you are probably right about the interpretation. ;)

So if sleep is not the right tool for the job, what would it be? And why does the EV (my $w2 = EV::periodic( 0, 1, 0, sub { print Math::BigFloat->new( sprintf("%d.%06d",gettimeofday) ), "\n";} );) seconds diff is greater than the original code (below is simplified version of the original code; on my computer it has diff bellow 0.0004, sometimes the avg is 0.0002)?

use strict;
use warnings;
use Time::HiRes qw/time sleep/;

sub negative_timer {
    my $max = $_[0];
    my @time_test;
    my $start = time();
    for (my $i = 0; $i < $max; $i++) {
        $start += 1;
        sleep $start - time();
        $time_test[$i] = time();
    }
    return @time_test;
}
[download]

I am running chronyd service, but not running ntpd

Yes, AFAIK the two are mutually exclusive. I updated my node here to suggest some of the chrony configuration directives to look at that might be relevant for clock jumps.

So if sleep is not the right tool for the job, what would it be?

I checked, and interestingly, libev uses nanosleep(2) (or select(2)), which is apparently quite similar to what Time::HiRes's usleep does - so the method actually isn't as fundamentally different from your approach as I initially thought (but Perl's built-in sleep is of course too inaccurate). However, in libev, the calculations for the next event time happen at the C level, so I think it could be a little bit more accurate than if you do the same on the Perl level.

And why does the EV ... seconds diff is greater than the original code (below is simplified version of the original code; on my computer it has diff bellow 0.0004, sometimes the avg is 0.0002)?

The difference might just be from inaccuracies in the calculation of the sleep time, or the time period over which you measured (e.g. there might have been different jitter during the two). I ran a test where I ran my code from here over almost an hour, with the addition that I wrote out the timestamp for each time the event fired. Although my gettimeofday on each event was on average 0.0017s after the whole second (max 0.0021), I could see no signs of a cumulative error - perhaps it would be best if you run your own tests over a longer period of time (like a day) to see if it suits your needs.

Update: Oh, and Average over 3312 deltas: 1.00001844328777194271613548903407496361, I'd hope you'd see this number shrink over an even longer test. Also minor edits.

Just save the value and test before sleeping.

use strict;
use warnings;
use Time::HiRes qw/time sleep/;

sub negative_timer {
    my $max = $_[0];
    my @time_test;
    my $start = time();
    for (my $i = 0; $i < $max; $i++) {
        $start += 1;
        my $dt = $start - time();
        if ($dt > 0) {
            sleep $dt;
            $time_test[$i] = time();
        }
    }
    return @time_test;
}
[download]

Thanks, Dean! I thought about doing that, but that would slow the calculation a bit, and thus the calculated second would differ a bit more.

Isn’t there a way to accomplish this by addition (incrementing the time by a second) instead of substraction (decreamenting the time by a second)?

Update

As I have said, after some tests, the average difference from a real second is -0.000003595544834, which is about 0.000003513663706 more than the average difference in the original code.

I know this is still somewhat irrelevant difference, but we need as precise second as we can get, and since the original version of the code is more precise (by that much), this new code does not solve our problem.

Instead of time(), have you tried clock_gettime(CLOCK_MONOTONIC) from Time::HiRes? The monotonic clock isn't adjusted by NTP and simply counts from some unspecified time in the past.

perl -MTime::HiRes=sleep -E 'say sleep 1'
[download]

Here you go. This is even worse.

$ time perl -MTime::HiRes=sleep -E 'say sleep 1'
1.000147

real    0m1.139s
user    0m0.006s
sys     0m0.008s
[download]

You miss the point of this test. Anonymous Monk's point is not to suggest calling out to an external perl. The point is that sleep() itself has inherently imprecise. You requested sleep(1) but actually got 1.000147 seconds of sleep. Therefore fussing over 0.00000XXXX seconds of "error" is futile.

Good Day,
    Dean

If the concern is first ensuring errors don't accumulate and second an emphasis on minimum variation in period then the following code should help:

use strict;
use warnings;
use Time::HiRes qw/time sleep/;

my $nextTick = int (time());

while (1) {
    $nextTick += 1.0; # We want to stuff again in 1 second

    my $wait = 0.99 - ($nextTick - time()); # allow 10ms slop in sleep
+ period

    sleep $wait if $wait > 0.0;

    1 while time() < $nextTick; # spin until time is up

    RunStuff();
}


sub RunStuff {
    # kick off required sub tasks here
    print time(), "\n";
}
[download]

The key is to calculate when the next interesting epoch is, then sleep for most of the time until then. The 1 while ... loop is a busy wait until the next epoch time has arrived.

Reading again, and guessing...

The problem is the cumulative error,
NOT the exact timing.

So, forget my hint at the cpu tick counter.
(Which would be the most exact "timer" at the average system..)

The solution is easy, just do (Pseudocode)


$oldtimestamp = timestamp();
nanosleep(0.9seconds);
while ( $oldtimestamp == timestamp() ) { 
  nop; 
}
spawnmyself();
dothemysterious();
[download]


This way in, say, 1 hour exactly 60*60 instances are going to be spawned.
good luck

Anyways, you should probably keep in mind, that you're about to write a fork bomb.

I see what you mean. Original code (not published here) had a check for that (see below) which is executed in the loop as launch(@ARGV). And if a Bash script should run periodically, we have a special Bash script to check if only one instance of a particular script is running. Although I must say, we are currently in the middle of porting of the Bash scripts to Perl.

sub launch {
    return if fork;
    exec @_;
    die "Couldn't exec";
}

$SIG{CHLD} = 'IGNORE';
[download]

For Perl scripts, we use the Highlander function to eliminate the forks. This, however, could effectively kill the processes we want to run each second (n seconds), when one instance is running longer that the period.

And generally, we need n * second intervals. Some scripts run each second, others every 5 seconds. I have created this question to find a way to run the scripts every n seconds most accurately.

Beside of this, I guess, best way for really exact timing would be spawning the script, which now wait's within a spinlock for the current second to change.

This is interesting to me, but I am not a Perl guru (not even a pro programmer), therefore I’d like to ask you for more info what exactly is the spawning and the spinlock (although for this I should open another question I think). Thanks. :)

You also don't write, which platform you're on.

We run the scripts (and Perl) on CentOS exclusively. No need for portability.

Save this as "something.hta" on your desktop and open it in Notepad. You will have to edit the location of the perl interpreter and the location of your perl script within the code. And then simply double-click on the JavaScript program and click on Start. This can also display the exact millisec when perl was executed, but I've noticed that it's not exactly 1 second. It's always 10 ms more. But you can adjust for that by calling the script every 990ms instead of 1000.

<HTML>
<BODY onLoad="Init();">
<INPUT TYPE=BUTTON VALUE="START" onClick="Start();">
<INPUT TYPE=BUTTON VALUE="STOP" onClick="Stop();">


<SCRIPT>

// I have tested this. It works.

function Init()
{
  WScript = new ActiveXObject("WScript.Shell");
  RUNNING = 0;
  document.title = "INACTIVE";
}

function RunPerl()
{
/*
  var T = new Date();
  document.title = T.getTime();  // SHOW MILLISECONDS IN THE WINDOW TI
+TLE BAR
*/
  WScript.Run('C:\\BIN\\PERL\\tinyperl.exe K:\\DESKTOP\\myscript.pl');
}

function Start()
{
  if (!RUNNING)
    RRR = setInterval(RunPerl, 1000);  // 1000 ms delay
  RUNNING = 1;
  document.title = "RUNNING";
}

function Stop()
{
  if (RUNNING)
    clearInterval(RRR);
  RUNNING = 0;
  document.title = "STOPPED";
}

</SCRIPT>
[download]