Re: lhs substr(): refs vs. scalars

You know you can use substr as an lvalue or with a fourth parameter to avoid duplicating your big scalars.

If you need to use unpack to decode small chunks of the buffer, or pack to overwrite small chunks, use them in conjunction with substr to avoid copying:

my @decoded = unpack '...', substr $bigscalar, $offset, $size;

substr $bigscalar, $offset, $size, pack '...', @newValues;

# or
substr( $bigscalar, $offset, $size ) = pack '...', @newValues;
[download]

But be very sure that the size specified in substr, and the size of the result from pack match exactly, otherwise you will be expanding or shrinking your big scalar by the difference which will lead to nasty surprises. It may be better to use an intermediary variable here:

my $replacement = pack '...', @newValues;

substr( $bigScalar, $offset, length $replacement ) = $replacement;
[download]

It is also possible (from 5.8.5 onwards) to set up an array of lvalue references to chunks of your scalar and then manipulate the individual chunks through indirection:

## Create a scalar
perl> $bigScalar = 'the quick brown fox jumps over the laxy dog';;

## Create an array of lvalue refs to the indivdual words using \substr
+...
perl> @lvrefs = map{ 
    \substr $bigScalar, $_->[0], $_->[1] 
} [0,3], [4,5], [10,5], [16,3], [20,5], [26,4], [31,3], [35,4], [40,3]
+;;

## Indirecting through the elements of the array gives you the words
perl> print $$_ for @lvrefs;;
the
quick
brown
fox
jumps
over
the
laxy
dog

## And assign through the elements allows you to replace them, in-plac
+e, individually
perl> ${ $lvrefs[7] } = 'lazy';; ## The ${ ... } is necessary.

## The typo is now corrected.
perl> print $bigScalar;;
the quick brown fox jumps over the lazy dog
[download]

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal?

"Science is about questioning the status quo. Questioning authority".

The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.

Comment on Re: lhs substr(): refs vs. scalars Select or Download Code

Replies are listed 'Best First'.
Re^2: lhs substr(): refs vs. scalars by renodino (Curate) on Oct 08, 2005 at 18:24 UTC
Thanks to all. I've learned much today, and will be revising a lot of scripts to avoid the param copying. I don't know how I've been YAPH'ing for 8 years wo/ knowing this 8^/. It will certainly be interesting to see what sort of performance boosts I get. Is it safe to assume the ref'ing of substr() will survive into future versions ?	[reply]
Re^3: lhs substr(): refs vs. scalars by BrowserUk (Patriarch) on Oct 08, 2005 at 18:57 UTC
I'm the wrong person to ask, but I would assume so, as they (substr refs) have steadily been corrected and improved over the last few versions. They have been available since before my time (5.6.1), but through a bug in the implementation, there was originally only 1 lvalue ref available at a time for each given string in te program. This was fixed in 5.8.5. The most useful use of them is processing fixed length record files where you allocate the input buffer and create an array of lvalue refs to the fields. You can now read or sysread subsequent records directly into the buffer overlaying the previous record, and the fields array now refers to the fields of the new record. It saves re-divvying the buffer over and over for each record, which can save a good deal of memory (re)allocation when processing large files. Add a few seeks and you have an efficient and fairly cache freindly way of doing in-place editing on huge, fixed record length files. Not they're much in vogue these days, but they do have their uses :). I played with manipulating huge tiff images like this one (Warning!!! 11,477 x 7,965 x 24 image 204MB) directly on disk. Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. Lingua non convalesco, consenesco et abolesco. -- Rule 1 has a caveat! -- Who broke the cabal? "Science is about questioning the status quo. Questioning authority". The "good enough" maybe good enough for the now, and perfection maybe unobtainable, but that should not preclude us from striving for perfection, when time, circumstance or desire allow.	[reply]
Re^4: lhs substr(): refs vs. scalars by renodino (Curate) on Oct 08, 2005 at 19:41 UTC
Actually, the segment ref's may be valuable for me. I'm using a pool of fixed buffers that get populated with binary msgs whose headers have a fixed format. So if I grab ref's to the header fields of interest when the buffers are created, my logic may be a bit simpler, and hopefully faster (tho I still need to pack/unpack(), so it may not be worth the effort). But still very good to know!	[reply]
Re^4: lhs substr(): refs vs. scalars by ysth (Canon) on Oct 09, 2005 at 03:45 UTC
Can you give a reference to the pre-5.8.5 bug? I recall there being a bug fixed where 3-arg substr misbehaved if the same call was used in both lvalue and non-lvalue mode, but that sounds like a different case than you describe.	[reply]
Re^5: lhs substr(): refs vs. scalars by BrowserUk (Patriarch) on Oct 09, 2005 at 04:33 UTC
Re^6: lhs substr(): refs vs. scalars by ysth (Canon) on Oct 09, 2005 at 04:40 UTC


laziness, impatience, and hubris
	PerlMonks