Beefy Boxes and Bandwidth Generously Provided by pair Networks
The stupid question is the question not asked
 
PerlMonks  

Why do $x="$h->{foo}" and $x=$h->{foo} have different effects (as reported by Devel::Peek)?

by ELISHEVA (Prior)
on Mar 25, 2009 at 16:29 UTC ( [id://753172]=perlquestion: print w/replies, xml ) Need Help??

ELISHEVA has asked for the wisdom of the Perl Monks concerning the following question:

I always thought an assignment was an assignment, but I guess I'm wrong. I wonder if anyone can explain to me why the following two sets of assignments generate different results for PV (when dumped via Devel::Peek)? This issue came up because, apparently Storable uses the PV value, not the IV value when freezing.while running some regression tests, I discovered that Storable was generating two different frozen strings for what appeared to be the same variable with the same assigned value. (See update below for more information)

$x = $hIn->{string}; $x = $hIn->{number}; Devel::Peek::Dump($x); #and $x = $hIn->{string}; $x = "$hIn->{number}"; Devel::Peek::Dump($x);

I might have thought that the datum assigned via direct assignment and interpolation might be different. But if that was so, why then do the following two bits of code also produce different results? We haven't (in Perl) actually assigned anything and yet the mere fact of using the variable in an interpolation seems to change it:

$x = $hIn->{string}; $x = $hIn->{number}; Devel::Peek::Dump($x); #followed immediately by print "$x\n"; Devel::Peek::Dump($x);

I've written a sample script to illustrate the differences and provided sample output from my machine (running Debian-Etch, Perl 5.8.8):

use strict; use warnings; use Devel::Peek; my $hIn = {string => 'abc', number => 5 }; print "#*** Setting...: " . q{$x=$hIn->{string}} . "\n"; my $x = $hIn->{string}; # we assigned a new value to $x BUT # Devel::Peek::Dump claims that $x has PV = "abc"\0 # !!!! ???? print STDERR "#*** " . q{$x = $hIn->{number};} . "\n"; $x = $hIn->{number}; Devel::Peek::Dump($x); # but after being used in an interpolation # Devel::Peek::Dump says that $x has PV = "5"\0 print STDERR "#*** " . q{print "<$x>\n"} . "\n"; print "<$x>\n"; Devel::Peek::Dump($x); print "#*** Resetting...: " . q{$x=$hIn->{string}} . "\n"; $x = $hIn->{string}; # we assigned a new value to $x BUT # Devel::Peek::Dump claims that $x has PV = "abc"\0 # !!!! ???? print STDERR "#*** " . q{$x = $hIn->{number};} . "\n"; $x = $hIn->{number}; Devel::Peek::Dump($x); # but after being used in an interpolation # Devel::Peek::Dump says that $x has PV = "5"\0 print STDERR "#*** " . q{"$x = $hIn->{number};"} . "\n"; $x = "$hIn->{number}"; Devel::Peek::Dump($x);

outputs the following:

#*** Setting...: $x=$hIn->{string} #*** $x = $hIn->{number}; SV = PVIV(0x8150b10) at 0x814f624 REFCNT = 1 FLAGS = (PADBUSY,PADMY,IOK,pIOK) IV = 5 PV = 0x81902e8 "abc"\0 CUR = 3 LEN = 4 #*** print "<$x>\n" <5> SV = PVIV(0x8150b10) at 0x814f624 REFCNT = 1 FLAGS = (PADBUSY,PADMY,IOK,POK,pIOK,pPOK) IV = 5 PV = 0x81902e8 "5"\0 CUR = 1 LEN = 4 #*** Resetting...: $x=$hIn->{string} #*** $x = $hIn->{number}; SV = PVIV(0x8150b10) at 0x814f624 REFCNT = 1 FLAGS = (PADBUSY,PADMY,IOK,pIOK) IV = 5 PV = 0x81902e8 "abc"\0 CUR = 3 LEN = 4 #*** "$x = $hIn->{number};" SV = PVIV(0x8150b10) at 0x814f624 REFCNT = 1 FLAGS = (PADBUSY,PADMY,POK,pPOK) IV = 5 PV = 0x81902e8 "5"\0 CUR = 1 LEN = 4

Many thanks in advance, beth

Update: added explanation of why this became an issue for me.

Update: corrected intro paragraph (with strikeout). With the help of kyle and ikegami I now have a much better understanding of the Devel::Peek output. It appears that my first impression (that Storable was ignoring flags and preferring PV was wrong. Storable does indeed choose between IV and PV correctly. The differences I was seeing was because it also seems to be storing the flags along with the value. So if something changes the flags the output of Storable::freeze(...) changes as well, even if the actual value stored does not. This doesn't appear to be a bug, but it is something to be aware of if one ever needs to work with frozen strings.

Once again, many thanks for the wonderful explanations all have provided. I have learned a great deal from this thread.

Best, beth

  • Comment on Why do $x="$h->{foo}" and $x=$h->{foo} have different effects (as reported by Devel::Peek)?
  • Select or Download Code

Replies are listed 'Best First'.
Re: Why do $x="$h->{foo}" and $x=$h->{foo} have different effects (as reported by Devel::Peek)?
by borisz (Canon) on Mar 25, 2009 at 16:52 UTC
    You use $x as number and sometimes as string. When you use it as number, IV holds the value and PV is updated on demand. Thats all.
    Boris

      Many thanks - I hadn't realized that PV was updated "just-in-time".

      But why doesn't this confuse Perl? After experimenting a bit, it seems that when IV and PV have different values, Perl always knows which one to trust - is it one of the flags? (Update: yes - see kyle's and ikegami's posts below)

      On the other hand, Storable seems to blindly favor PV when both IV and PV exist, even if they are in conflict. Is this a bug in Storable (v. 2.15)? Should it be doing what Perl does, but isn't?

      Best, beth

      Update:After further experimentation it appears that Storable does dump the correct value but its choice of binary representation for that value seems to be affected by the flags at the time of freezing. Thus an IV only variable (only IOK,pIOK flags set) freezes to a different string than a variable where both PV and IV are set (IOK,pIOK,POK,pPOK flags set). This is true even though both would stringify to the same Perl string (e.g. 5 => "5") and both would appear to be equal (in Perl) when thawed. That is, $x1 eq $x2, $x1 == $x2 would both be valid and true, even in strict mode.

        But why doesn't this confuse Perl?

        Using the output of your first snippet as an example,

        SV = PVIV(0x8150b10) at 0x814f6b4 <-- Can contain an IV and PV REFCNT = 1 FLAGS = (IOK,pIOK) <-- Only contains an IV IV = 123 PV = 0x81623f0 "abc"\0 CUR = 3 LEN = 4 SV = PVIV(0x8150b10) at 0x814f6b4 <-- Can contain an IV and PV REFCNT = 1 FLAGS = (POK,pPOK) <-- Only contains a PV IV = 123 PV = 0x81623f0 "123"\0 CUR = 3 LEN = 4

        The type of the scalar is promoted as necessary. Downgrading would be a waste of time, so flags are used to indicate what kind of value the scalar contains.

        Note that it's possible to have a scalar that has both an IV and PV.

        $ perl -MDevel::Peek -e'$x = 123; Dump $x; "$x"; Dump $x' SV = IV(0x816a41c) at 0x814f69c REFCNT = 1 FLAGS = (IOK,pIOK) <-- Only contains an IV IV = 123 SV = PVIV(0x8150b10) at 0x814f69c REFCNT = 1 FLAGS = (IOK,POK,pIOK,pPOK) <-- Stringification has IV = 123 been cached. PV = 0x81651a0 "123"\0 CUR = 3 LEN = 4

        On the other hand, Storable seems to blindly favor PV when both IV and PV exist,

        When both IV and PV exist (i.e. when the scalar is a PVIV or subtype), or when POK is set? The former would definitely be a bug. Otherwise, read on.

        If a scalar contains both a number and a string (POK and either IOK or NOK), the situation is intended to be one of the following (and almost always is):

        • The string is a stringification of the IV.
        • The string is a stringification of the NV.
        • The number is a numification of the string.

        If we accept that more general dualvars can't be stored, we should be able to store only one of the string or the number. So which one should we used when?

        The string is a stringification of the IV.

        Stringification of IVs is lossless, and so is their numification.

        Either the string or the IV will do.
        The string is a stringification of the NV. Stringification of NVs is practically lossless, and so is their numification. Either the string or the NV will do.

        The number is a numification of the string.

        Numification of non-numeric strings is lossy. e.g 0+"abc" ne "abc".

        Numification of partially numeric strings is lossy. e.g 0+"123abc" ne "123abc".

        Numification of some numeric strings is lossy. e.g 0+"0.50" ne "0.50".

        Necessarily, we need to store the string.

        As such, always storing the string is sufficient and appropriate.

        Update: Storing the string instead of the number also allows us to store '0 but true' and '0E0', true values that evaluate to zero.

Re: Why do $x="$h->{foo}" and $x=$h->{foo} have different effects (as reported by Devel::Peek)?
by derby (Abbot) on Mar 25, 2009 at 17:06 UTC

    You're just seeing the effects of perl's internals and how SVs can have multiple values and context is how perl determines which one to use (and assignment does not overwrite the entire SV but just that particular context). You can have fun with this with Scalar::Util's dualvar:

    use Devel::Peek; qw( Dump ); use Scalar::Util qw( dualvar ); my $foo = dualvar 10, "Hello"; Dump($foo); print $foo, "\n"; print $foo + 5, "\n";

    -derby

    Update: And way down in the Storable pod is:
    By default Storable prefers to store a scalars string representation (if it has one)
    so ... yeah, that's going to give you problems with dualvars.

Re: Why do $x="$h->{foo}" and $x=$h->{foo} have different effects (as reported by Devel::Peek)?
by kyle (Abbot) on Mar 25, 2009 at 17:15 UTC

    I'm no expert on Perl internals, so take what I say with some salt.

    That said, I think I have an idea about what's going on here. A scalar in Perl can have a number or a string in it, and Perl will convert back and forth between them as necessary. The structure that Perl uses for a scalar has a pointer for a string and a pointer for a number, and it has flags to show which are valid representations of this scalar. That way, if you stored a string, and it converted it to a number, it can hold both of them and know that they're both valid and not have to do the conversion again. When you store a string or a number, the other pointer is marked as invalid but not actually changed.

    You can see this in your tests. The pointers to values are IV and PV for numbers and strings, respectively. The flags for their validity are p?IOK and p?POK.

    Lets look at the test.

    1. You create a string and a number.
    2. You set a new $x to the string. That sets its PV.
    3. You set $x to the number. That sets the IV and invalidates the PV.
    4. You peek at $x and see it has both IV and PV set to unrelated values (number and string), but only one is valid—IOK (the number).
    5. You use $x as a string. Perl converts the number to a string and stores it as the PV and flags the string representation as valid.
    6. You peek again and see IV and PV both as 5-ish and both have flags to show them as valid (POK and IOK).
    7. You set $x to the string value. The IV remains 5, but the PV changes to the new value. The IOK flag is removed, so the stale number (5) won't be used.
    8. You set $x to the number. The PV remains the string, and the IV "changes" to the value it had anyway. The flags get flipped so that IOK is true and POK is false. Now $x can be used as-is as a number, but the string representation is invalid.
    9. You peek again. IV = 5, PV = abc, but only IOK.
    10. You assign to $x the number interpolated into a string. That changes the number to a string and gives the string value to $x. The PV is set to "5". IV is coincidentally also 5, but it's not flagged as valid because a string was just assigned.
    11. One last peek to show what I just described.

    The only time this string/numeric duality ever bit me was the time I documented in Strings and numbers: losing memory and mind.. Long story short, having both representations of something laying around takes a little extra memory, and I had a lot of scalars with both values set.

Re: Why do $x="$h->{foo}" and $x=$h->{foo} have different effects (as reported by Devel::Peek)?
by Anonymous Monk on Mar 26, 2009 at 02:52 UTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://753172]
Approved by Corion
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others imbibing at the Monastery: (1)
As of 2024-04-25 01:25 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found