Module callbacks - to fork or not fork

Bod has asked for the wisdom of the Perl Monks concerning the following question:

I am writing a module to handle Stripe webhook calls. The module does the necessary checking that the call has come from Stripe and provides the user with a means to give defined callbacks for each webhook. They simply pass in a parameter that matches the Stripe event so it is compatible when Stripe adds new events.

The module is instantiated like this:

my $stripe = Stripe::Webhook->new(
    'signing_secret'    => 'whsec_xxxxxxxxxx',
    'invoice-paid'      => \&paid,
    'all-webhooks'      => \&all,
);
[download]

Here we call &paid; when Stripe sends an invoice.paid event. Plus, we call &all; for every event sent.

The person using the module could do things that take a significant time in those subs. But the documentation says "your endpoint must quickly return a successful status code (2xx) prior to any complex logic that could cause a timeout.".

So should I fork to another process for the callback or should I warn the user of the module in the documentation? Something along the lines of: "If your logic might take some time to complete, fork a new process and perform your logic there. This will allow a timely reply to be sent back to Stripe."

Or should I be dealing with this problem in some other way?

This is the code that provides the callback:

my $hook_type = $self->{'webhook'}->{'type'};
$hook_type =~ s/\./-/g;
if (defined &{$self->{$hook_type}}) {
    $self->{'reply'}->{'status'} = 'success';
    &{$self->{$hook_type}}($self->{'webhook'});
}
[download]

In my own implementation of handling events from Stripe, I only make a couple of calls to the database and write to a text file. But of course, I have no idea what other people might want to do with the callbacks. They might decide to send an email which is usually not exactly quick...

Comment on Module callbacks - to fork or not fork Select or Download Code

Replies are listed 'Best First'.
Re: Module callbacks - to fork or not fork by NERDVANA (Deacon) on Mar 01, 2023 at 05:05 UTC
As general advice for authoring modules, try not to get too wrapped up in the potential problems of users. Let them know about the timing requirements and they can worry about post-processing. There's quite a few ways to do it, too: forking, threads, event libraries, or Minion. But also, regarding send an email which is usually not exactly quick I'm going to jump in with related advice that when a web app wants to generate an email as part of a transaction with the user, the email should be added to a queue table of the database, to ensure the event is initiated. Then, come back with a background job that pushes the email from the table to SMTP or Sendgrid or whatever. This way 1) you never lose emails, and 2) service interruptions in SMTP or SendGrid will never delay the HTTP response to the user. You also gain a lot of control over how you handle the errors, like alarms that go off any time the table has stale messages in it, or automatic retries within the first 10 minutes, etc.	[reply]
Queueing emails (was: Re^2: Module callbacks - to fork or not fork) by Bod (Parson) on Mar 01, 2023 at 23:11 UTC
I'm going to jump in with related advice that when a web app wants to generate an email as part of a transaction with the user, the email should be added to a queue table of the database, to ensure the event is initiated. Then, come back with a background job that pushes the email from the table to SMTP or Sendgrid or whatever. Thanks for pointing that one - I wholeheartedly agree. I now do exactly this but, until recently, I didn't. I'm sure lots of people don't just because it had never occurred to them. There is a third advantage to add to your list. It is much easier to change the way you send emails. I've started using Send In Blue over SendGrid or SMTP. When I switched a system over from using MIME::Lite, making the change was simple as it only has to happen in one sub of one module.	[reply]
Re^2: Module callbacks - to fork or not fork by Bod (Parson) on Mar 01, 2023 at 23:18 UTC
As general advice for authoring modules, try not to get too wrapped up in the potential problems of users. Let them know about the timing requirements and they can worry about post-processing That seems very reasonable... I feel that is right approach to take - thanks	[reply]
Re: Module callbacks - to fork or not fork by stevieb (Canon) on Mar 01, 2023 at 02:46 UTC
Do the things executing inside the callbacks need to be completed successful before you send back your `2xx` status code? If so, then forking won't help because you need to know if those subs succeeded or not.	[reply] [d/l]
Re^2: Module callbacks - to fork or not fork by hippo (Bishop) on Mar 01, 2023 at 10:19 UTC
Contrarily, if the callbacks don't need to be completed beforehand then you don't necessarily need to fork either: just send your 200 first and then work through the callbacks. 🦛	[reply]
Re^3: Module callbacks - to fork or not fork by Bod (Parson) on Mar 01, 2023 at 23:16 UTC
just send your 200 first and then work through the callbacks Why didn't I think of that??? I guess I'm conditioned to doing the logic first, then the output... I've not tested it, but could there be a danger that the 200 and its payload will get buffered and will end up waiting after the callbacks complete? I have written code before that outputs HTML to a browser and then does other time-consuming stuff. But the HTML gets stuck in a buffer and I see a 408 timeout error instead of the HTML.	[reply]
Re^4: Module callbacks - to fork or not fork by hippo (Bishop) on Mar 02, 2023 at 23:15 UTC
Re: Module callbacks - to fork or not fork by Anonymous Monk on Mar 01, 2023 at 10:00 UTC
I stumbled reading the code fragment and had to consult the manuals (thanks for that). Are there any benefits of `'defined &$maybe_coderef`' syntax (over `exists` or `'CODE' eq ref...` or TIMTOWTDI-others)? It triggers 'uninitialized' warning for uninitialized value, but maybe there are not obvious benefits?	[reply] [d/l] [select]
Re^2: Module callbacks - to fork or not fork by hippo (Bishop) on Mar 01, 2023 at 10:27 UTC
TIMTOWTDI-others With Ref::Util you can test it safely, quickly and all-in-one with `is_coderef ($maybe_coderef)`. If you have not already looked at Ref::Util I can wholeheartedly recommend it. Your code will be shorter, less error-prone, faster and arguably more readable/maintainable with it. 🦛	[reply] [d/l]
Re^3: Module callbacks - to fork or not fork by Bod (Parson) on Mar 01, 2023 at 23:06 UTC
If you have not already looked at Ref::Util I can wholeheartedly recommend it But Ref::Util is not core...	[reply]
Re^4: Module callbacks - to fork or not fork by kcott (Archbishop) on Mar 02, 2023 at 02:19 UTC
Re^5: Module callbacks - to fork or not fork by Bod (Parson) on Mar 02, 2023 at 10:46 UTC
Some notes below your chosen depth have not been shown here


Your skill will accomplish what the force of many cannot
	PerlMonks