Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Best way to allocate some memory first

by Marshall (Canon)
on Dec 03, 2021 at 22:11 UTC ( #11139378=note: print w/replies, xml ) Need Help??


in reply to Best way to allocate some memory first

I would not consider 41 KB as "large".

Some years ago I spent literally days benchmarking and working with pre-sizing hash structures. Perl starts off with 8 hash "buckets". Every time that the hash doubles in size, 8,16,32,64, buckets etc. Perl has to re-visit every item in the hash to find it's new "hash index" (that's an entry in the hash table which is a pointer to a linked list).

My conclusion was that pre-sizing a hash table to hold say 100K hash keys didn't matter at all in the scheme of things. I don't remember all of the initial "bucket array" sizes that I experimented with.

A major factor in my analysis was, that the time spent actually using the hash completely dwarfed the time spent creating it in the first place. I took out the pre-sizing because it made no significant application perceptible difference.

Perl itself will call low level C functions to do its internal data structure management. These are "very fast" functions. Yes, I can write an ASM function that will beat C memcpy(), but so what? (memcpy()is good C code, it uses byte operations to align onto a memory boundary and then proceeds from there. The compiler, except at extreme optimization levels, cannot recognize that there is a specialized ASM instruction that is much faster at these block memory operations.) Again, so what?

Anyway, I am very skeptical that "pre-sizing a 41K Perl array" will make any difference at all in terms of overall application performance.

Update: I don't think that this will achieve the intended purpose, because undef is a value. This will mess things up when trying to decide how many elements are in the array now.
I don't know what Perl "magic" is. But I would guess that when your program initializes, if you use a loop to set every element to 1 and then use a loop to set every element to undef, that you will wind up with an array with memory allocated for each element. You can't even blink as fast as this will happen. Benchmark the heck of this and let us know what happens.

  • Comment on Re: Best way to allocate some memory first

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://11139378]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (1)
As of 2022-05-19 01:28 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?
    Do you prefer to work remotely?



    Results (71 votes). Check out past polls.

    Notices?