Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Re^6: references--hard vs anonymous operational weirdness

by ikegami (Patriarch)
on Mar 23, 2008 at 04:41 UTC ( [id://675734]=note: print w/replies, xml ) Need Help??


in reply to Re^5: references--hard vs anonymous operational weirdness
in thread references--hard vs anonymous operational weirdness

As you can see the same address is being reused.

Even if it wasn't optimized, that wouldcould be the case too.

In the non-optimized world, the my variable would get freed at the end of the for loop (since there are no longer any references to it) and reallocated at the top. Logically, the memory manager wouldcould find the same free block the second time around as the first since nothing changed.

While there is an optimization involved here to save some memory management work, it doesn't cause any change in the observed behaviour. What you call an optimization is just garbage collection, not an optimization at all.

Updates marked using INS and DEL elements. See chromatic's reply for details.

Replies are listed 'Best First'.
Re^7: references--hard vs anonymous operational weirdness
by chromatic (Archbishop) on Mar 23, 2008 at 05:44 UTC
    Logically, the memory manager would find the same free block the second time around as the first since nothing changed.

    Not necessarily; plenty of garbage collectors try to make object creation as cheap as possible, so the memory manager could allocate a new SV header out of the same pool every time (bumping up the pointer to the next free object in the pool by one unit every time) instead of maintaining a free list of SV headers. If you can get a copying/compacting collector to work correctly (moving headers can be somewhat tricky if you don't want to stop the world), you avoid mucking with a free list. In my experience with Parrot, you actually lose time with the free list games when you malloc() a new pool and have to stuff every header there into the free list.

    Granted, Perl 5 doesn't do any of this, and your analysis of what happens, why this isn't an optimization, and why hipowls's comment absolutely baffled me is completely correct.

    Still, speaking "logically" about a memory manager isn't always necessarily possible. There's a lot of subtlety possible there. (I didn't even mention the cache SV attached to some ops in Perl 5.)

      Quite true. I was way more definite than I meant to be.
Re^7: references--hard vs anonymous operational weirdness
by hipowls (Curate) on Mar 24, 2008 at 10:02 UTC

    Thanks for your reply. I now understand where I went wrong. Just to try and reduce the degree of bafflement going round I'll explain my (admittedly erroneous) thinking.

    On each iteration of the loop the (simplified) process is

    1. Allocate memory for @array
    2. Initialize @array
    3. Deallocate memory assigned to @array
    Note: I am only talking about the memory allocated for @array (the AV struct), not the value of @array (the contents).

    A smart compiler can optimize that loop so that the allocation and deallocation of the memory occur outside the loop, that is only once. Based on my observation of the actual memory allocation I had assumed that perl was optimizing the loop in just such a manner.

    Perl, however, is not that simple and its loop would be more like
    1. Allocate memory for @array
    2. Initialize @array
    3. Deallocate memory assigned to @array if not referred to else where
    I am speculating that means that perl can't decide until runtime if the memory slot can be reused which in turn means the optimization isn't possible, at least in the simplest form and may not be worth the overhead in any case.

    So what I was seeing was a chunk of memory deallocated and then an immediate request for a chunk of the same type and the memory manager handed over the chunk just released.

      On each iteration of the loop the (simplified) process is

      1. Allocate memory for @array
      2. Initialize @array
      3. Deallocate memory assigned to @array

      Well, I don't know what you mean by "Initialize @array", but otherwise, that's the documented behaviour. And in most cases — including the one you called optimized — it's the observable behaviour.

      Perl, however, is not that simple and its loop would be more like

      1. Allocate memory for @array
      2. Initialize @array
      3. Deallocate memory assigned to @array if not referred to else where

      No, that's not what it does. For starters, the @array is never deallocated by the loop (or sub).

      1. Allocate memory for @array if it isn't already allocated. (I don't actually know when this is done. It might be earlier.)
      2. When my is executed, put a directive on the stack to clear/free @array.
      3. On stack unwind, in response to the directive, clear the @array if there are no more references to it. Otherwise, allocate a new @array.

        I was thinking of allocation to be handing over a chunk of raw memory and initialization to be things like setting ref count to 1, the pointer to the contents to NULL or however an empty list is represented and any other bookkeeping. It is possible that these two steps are a single opcode and can't be separated.

        On stack unwind, in response to the directive, clear the @array if there are no more references to it. Otherwise, allocate a new @array.
        Surely allocation of a new array would only be done on entry to the block at the start of each iteration?

        Once more thanks for your time. If you would care to point me to documentation where I can read some more that would be helpful (and spare you more tedious questions;). I am looking for somewhere between perl internals and the source tarball. Even knowing where to start on the source would be useful.

        Just spotted perl -lne 'print if /^[^\/]+\.[ch]\s+/' MANIFEST in perlhack, I guess that's where I start.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://675734]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others admiring the Monastery: (5)
As of 2024-03-28 20:17 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found