http://qs321.pair.com?node_id=1214779


in reply to Re^6: Abusing Map
in thread Abusing Map

The problem comes when you need to build a list of indexes as with the OPs code:

sub checkMem{ `tasklist /nh /fi "pid eq $_[0]"`; };; print checkMem( $$ );; perl.exe 3040 Console 1 +9,544 K $#a = 1e6;; print checkMem( $$ );; perl.exe 3040 Console 1 1 +7,428 K map{ $_ } 0 .. $#a;; print checkMem( $$ );; perl.exe 1224 Console 1 7 +3,272 K ### 56MB to build the list of 1e6 indices ###

With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
In the absence of evidence, opinion is indistinguishable from prejudice. Suck that fhit

Replies are listed 'Best First'.
Re^8: Abusing Map
by AnomalousMonk (Archbishop) on May 17, 2018 at 19:19 UTC
    The problem comes when you need to build a list of indexes ...

    Ah, ok, I see now what you're talking about.


    Give a man a fish:  <%-{-{-{-<

Re^8: Abusing Map
by Anonymous Monk on May 17, 2018 at 19:31 UTC

    I remember reading that for ($start..$end) kind of loops are also optimized not to produce an intermediate list:

    $ perl -s -e' sub mymem { system("ps -q $$ -o vsz") }; mymem(); $#a = $size; mymem(); $a[$_]=0 for 0..$#a; mymem() ' -- -size=1e7 VSZ 18724 VSZ 96852 VSZ 333528 $ perl -s -e' sub mymem { system("ps -q $$ -o vsz") }; mymem(); $#a = $size; mymem(); $a[--$size]=0 while $size; mymem() ' -- -size=1e7 VSZ 18724 VSZ 96852 VSZ 333528 $ perl -E'say $^V' v5.24.1
    At least, they seem to be on my machine.

    This doesn't cover map, though:

    $ perl -s -e' sub mymem { system("ps -q $$ -o vsz") }; mymem(); $#a = $size; mymem(); map {$_} 0..$#a; mymem() ' -- -size=1e7 VSZ 18724 VSZ 96852 VSZ 573664

      From PerlOP:
      In the current implementation, no temporary array is created when the range operator is used as the expression in foreach loops, but older versions of Perl might burn a lot of memory when you write something like this:

      Many of my efficiency habits formed back in 5.6/5.8 days, and I've never seen the need to change them.

      One problem is that optimisation only kicks in in certain circumstances -- and I don't remember if I ever knew what they exactly were. I have a sneaking suspicion that in 32-bit perls, anything over 2**31 may have caused a list, but I haven't used a 32-bit perl for a decade, so I cannot check.

      One example that bit me was the iterator moving outside the range of integers:

      1 for 1e200 .. 1e200+1e6;; [Range iterator outside integer range at (eval 10) line 1, <STDIN> lin +e 2.

      Easy enough to re-write that one, but not so much if the numbers come from outside or are generated.

      The while/until loop approach handles anything 64-bit floats can, and (from) memory, is usually faster than postfix for even for integers; though that could have changed a lot since I last benchmarked it.

      I like to program in a consistent way -- where it doesn't compromise performance to do so -- and tend to stick with patterns once I've established ones that work.

      Eg. I still use '<:raw' as a matter of course, even though things changed and it was deprecated when they re-vamped PerlIO, and it was demonstrated that some other combination of pushing, popping and/or applying of layers was apparently faster. The whole thing just got too complex to remember and too convoluted to test.

      The whole PerlIO thing was something of a debacle IMO; a vast amount of effort and change for something that 95% of Perl users have never used :(


      With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
      Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
      "Science is about questioning the status quo. Questioning authority". The enemy of (IT) success is complexity.
      In the absence of evidence, opinion is indistinguishable from prejudice. Suck that fhit