Re: Substitution with regex and memory consumption

In general after a successful match (or the match part of a substitution), the regex engine keeps a copy of the original string so that it can dynamically generate values for $1, $2, $&, $` etc on demand. This string needs to be kept for at least as long as the surrounding scope - i.e. the scope of $1 etc. The details are far more complex, but internally perl's Copy-on-Write mechanism often (but not always) avoids having to do a real copy. But it doesn't always work out for the most efficient use of memory.

Dave.

Comment on Re: Substitution with regex and memory consumption

Replies are listed 'Best First'.
Re^2: Substitution with regex and memory consumption by k-mx (Scribe) on Mar 01, 2020 at 09:19 UTC
Okay, thank you! Some assumptions, please correct me if i'm wrong: Prior 5.18.0, s/// will copy original only if one of these was set: $&, $`, $'. Interpreter will set global PL_sawampersand flag that can't be disabled later. m// is also affected by this flag. Between 5.18.0 and 5.20.0, Perl can track usage of mentioned variables separately and copy only requested part of string. Perl 5.20.0+, successful s/// match always changes string, so COW mechanism always had to copy original. So, before 5.20 we have choice: avoid $&, $`, $' and use /p modifier to explicitly copy ${^*MATCH}. Now we can use $&, $`, $', m// don't suffer from PL_sawampersand anymore, but s/// will always copy original string, PL_sawampersand state doesn't matter, and nothing we can do with that.	[reply]
Re^3: Substitution with regex and memory consumption by dave_the_m (Monsignor) on Mar 01, 2020 at 22:47 UTC
That's roughly it, yes. Dave.	[reply]
Re^4: Substitution with regex and memory consumption by k-mx (Scribe) on Mar 08, 2020 at 08:16 UTC
Isn't this behavior incorrect? Why we must copy variable on each substitution? I think we can: respect PL_sawampersand and /p flag for substitution and have only one copy for string (last one) in current scope.	[reply]
Re^5: Substitution with regex and memory consumption by dave_the_m (Monsignor) on Mar 08, 2020 at 11:51 UTC


Keep It Simple, Stupid
	PerlMonks