eval makes JSON::XS even slower than JSON::PP

seriously - it's 10 to 100 times slower to use eval() rather than a native decode_json() in a mod_perl production webserver on non-trivial data

Your experience seems to differ from everyone else's experience. This is probably why all the JSON module authors decided it was fine to use croak. Please show some kind of benchmark if you want to convince anyone.

I just ran a benchmark (command line, on a laptop, perl 5.34) and see that for extremely small json decodes of 1 object with one key, the eval is as high as 20% overhead on the fastest XS modules (but still 2.5+ million per second). It's a ~0% difference for JSON::PP (mine actually ran the eval faster than without, but that probably indicates the noise level in the benchmark as a whole). This assumes valid json - of course for invalid JSON, triggering exceptions would be slower than a parser that returns undef, but if you're worried about denial of service, I think you'll run out of bandwidth before running out of processor to catch the exceptions.

For extremely large JSON texts, the eval overhead is lost in the noise. (essentially no difference)

$ perl benchmark-eval.pl 
long_json: 2468099
            (warning: too few iterations for a reliable count)
                    Rate pp_small pp_small_eval xs_small_eval cxs_small_eval xs_small cxs_small
pp_small         77101/s       --           -2%          -96%           -96%     -97%      -97%
pp_small_eval    78309/s       2%            --          -96%           -96%     -97%      -97%
xs_small_eval  2083333/s    2602%         2560%            --            -4%     -15%      -17%
cxs_small_eval 2173913/s    2720%         2676%            4%             --     -11%      -13%
xs_small       2439024/s    3063%         3015%           17%            12%       --       -2%
cxs_small      2500000/s    3142%         3092%           20%            15%       3%        --
                 Rate pp_long_eval pp_long cxs_long_eval cxs_long xs_long_eval xs_long
pp_long_eval  0.596/s           --     -0%          -98%     -98%         -98%    -98%
pp_long       0.597/s           0%      --          -98%     -98%         -98%    -98%
cxs_long_eval  24.4/s        3993%   3983%            --      -1%         -18%    -18%
cxs_long       24.5/s        4018%   4008%            1%       --         -18%    -18%
xs_long_eval   29.9/s        4909%   4897%           22%      22%           --      0%
xs_long        29.9/s        4909%   4897%           22%      22%           0%      --

Here's my benchmark: feel free to adjust it in such a way that it shows your 10x - 100x slowdown, and post for us to look at.

use v5.20;
use warnings;
use Benchmark qw( :hireswallclock cmpthese );
use JSON::PP ();
use JSON::XS ();
use Cpanel::JSON::XS ();

my $json_pp= JSON::PP->new;
my $json_xs= JSON::XS->new;
my $json_cxs= Cpanel::JSON::XS->new;
my $small_json_string= q|{"test":1}|;
sub random_data {
  return { map { rand() => $_[0]? random_data($_[0]-1) : rand() } 1..4
+0 }
}
my $long_json_string= $json_xs->encode(random_data(2));
say "long_json: ".length $long_json_string;

cmpthese(1000000, {
  pp_small_eval  => sub { eval { !$json_pp->decode($small_json_string)
+ } },
  xs_small_eval  => sub { eval { !$json_xs->decode($small_json_string)
+ } },
  cxs_small_eval => sub { eval { !$json_cxs->decode($small_json_string
+) } },
  pp_small       => sub { !$json_pp->decode($small_json_string) },
  xs_small       => sub { !$json_xs->decode($small_json_string) },
  cxs_small      => sub { !$json_cxs->decode($small_json_string) },
});

cmpthese(40, {
  pp_long_eval  => sub { eval { !$json_pp->decode($long_json_string) }
+ },
  xs_long_eval  => sub { eval { !$json_xs->decode($long_json_string) }
+ },
  cxs_long_eval => sub { eval { !$json_cxs->decode($long_json_string) 
+} },
  pp_long       => sub { !$json_pp->decode($long_json_string) },
  xs_long       => sub { !$json_xs->decode($long_json_string) },
  cxs_long      => sub { !$json_cxs->decode($long_json_string) },
});
[download]

Edit: Removed my comment about threaded/nonthreaded perl, since it was a strawman argument.

Try 'strings' instead if sub{}s for less noisy benchmark

I'm the one pointing out that you're making a wild assertion (paraphrasing, "BLOCK eval causes an order of magnitude(s) slowdown") which runs counter to consensus experience without giving any concrete evidence or providing any sample code which evidences the problem in question by myself providing a counter example, yes.

Show us code which evidences this purported order of magnitude(s) slowdown and I bet you'll get a reply from someone either pointing out what you're doing wrong, or if it's actually something you'll have eyes look at it from someone quite likely to be able to fix the problem in perl itself. BLOCK eval does add a small overhead because of the extra steps involved, but not that much (it's a couple of extra opcodes in the tree that don't really do much of anything if no exception occurs).

Or keep whinging about downvotes because you posted something detail- and code-free that (observationally) most people reading think is bunk. Tomayto, tomahto.

Edit: Tested the same benchmarks with a 700+k character "NON-TRIVIAL" chunk of both valid and invalid JSON (the later made by substituting in random garbage somewhere in the former). Aside from the processing rate understandably going down with the larger amount to parse the results look similar showing that the presence or absence of eval has no measurable effect (hundreds/sec for the non-PP versions versus 30k/sec; PP gets on the order of 1/sec eval or no).

> a wild assertion (paraphrasing, "BLOCK eval causes an order of magnitude(s) slowdown") which runs counter to consensus experience

My guess:

The OP was highlighting a modperl environment, which is notorious for global side effects IIRC.

So if one of the other modperl clients is manipulating the error mechanism, (like setting $SIG{WARN} or CGI::Carp or something homebrewn), this could lead to some slowdown.(Disclaimer: I never tried this, especially never tried modperl)

It's hard to tell why this is triggered by eval {BLOCK} , but this could be connected to ignoring that special variable which indicates the handler was fired inside an eval. (See $^S in perlvar )

All of this is of course not related to JSON ... and switching to another technology like fastcgi or PSGI should fix it.

Tho I wouldn't be surprised if the OP fiddled with error handlers by himself without taking care of $^S

Cheers Rolf
_{(addicted to the Perl Programming Language :)
Wikisyntax for the Monastery}

Dave.

I did benchmark it properly.

An empty claim when you could show your benchmark.