Re^2: some efficiency, please

in reply to Re: some efficiency, please
in thread some efficiency, please

Thanks. There are just so many tricks I never thought of (like building the entire array into the qr statement). :)

Would have liked to have used map, but the actual data is more like:

 ref 1
 ref 2
1 begin
end
2 begin
bar
end
...
[download]

with the numbers outside (right before) the beginning of each paragraph. I am sure that can be done with map, but it is a little too tricky for me at my level. :)

I killed what I had before after thirteen hours of CPU time, but I guess it (eventually) would have finished. I started it again with a print statement right before it removes each line from the file, and it starts out very quickly (just a few seconds per line removed), and then just keeps slowing down (after about a half-hour, it was well over a minute per line removed).

Still don't understand why just changing the one line:

for (@objects) { $allfile =~ s/^ +$_\n//m; }
[download]

to:

for (@objects) { $allfile =~ s/^ *(foo)? +$_\n//mn; }
[download]

caused it to slow down SO much.

Anyway, I was satisfied with the performance I had before (without the test for foo), but your method is an order of magnitude faster than that "without the test" method, and it (of course) catches the rare case when foo is there, so I am extremely grateful for that. Thanks, again.

Comment on Re^2: some efficiency, please Select or Download Code

Replies are listed 'Best First'.
Re^3: some efficiency, please by dsheroh (Monsignor) on Apr 14, 2019 at 09:09 UTC
Still don't understand why just changing the one line... caused it to slow down SO much. Just off the top of my head, I suspect it may be a bad interaction between the " ``" and the " `+`" causing the regex engine to backtrack excessively because they're separated only by an optional element, so a string of multiple spaces can match in multiple ways (no spaces/5 spaces, 1 space/4 spaces, etc.) which then multiplies the number of potential matches for the full string, each of which needs to be evaluated until the engine is satisfied that it either found one that's good enough or that no match exists. The capturing parens on `foo` may also be contributing. If you want to test this theory, you could try changing the regex to `$allfile =~ s/^( foo)? +$_\n//mn;` (leaving the capturing parens intact) or `$allfile =~ s/^(?: *foo)? +$_\n//mn;` (non-capturing parens, since you're really only using them for grouping) and seeing if that restores the original performance.	[reply] [d/l] [select]

Replies are listed 'Best First'.

Re^3: some efficiency, please
by dsheroh (Monsignor) on Apr 14, 2019 at 09:09 UTC

Still don't understand why just changing the one line... caused it to slow down SO much.

*

+

foo

If you want to test this theory, you could try changing the regex to $allfile =~ s/^( *foo)? +$_\n//mn; (leaving the capturing parens intact) or $allfile =~ s/^(?: *foo)? +$_\n//mn; (non-capturing parens, since you're really only using them for grouping) and seeing if that restores the original performance.

[reply]
[d/l]
[select]

In Section Seekers of Perl Wisdom