http://qs321.pair.com?node_id=445927

Recently, while writing a couple of nodes on references (they were intended to be "instructive" though I'm afraid they turned out to be just tiring), I struggled with the question of whether to bring up the subject of autovivification, and if so how thoroughly. Since the nodes were already bloated, I punted, regretfully.

And, as it happens, it seems that this is the tack that most expositions of Perl references take. When autovivification is mentioned at all, it is to rave about its (undeniable) virtues, i.e. the Good. It's as if we are so eager to encourage the tremulous newbie to try riding the bicycle without the training wheels, that we don't want to dampen any enthusiasm with talk of potholes, and semis. Indeed, only rarely is the dark side of autovivification mentioned, let alone discussed at any length (hence the non-standard ordering of adjectives in this node's title, although I also like the appropriateness of the acronym that results from this reordering). This means that the new programmer, happily rolling along, secure in Perl's dwimitude, usually learns about the Bad and the Ugly sides of autovivification by crashing against a nasty bug.

That's what happened to me...

I was a much younger programmer then...(insert your favorite flashback effects here). The software I was writing was for doing some statistical analysis on large collections of entities and their attributes (also numerous). I was using "association tables", that were implemented as HoHs whose primary keys were entity ids and secondary keys were attribute ids (or viceversa). (I was using undef as the value of all these ordered pairs, which probably didn't help.)

Here's the Bad. Consider the following snippet:

use strict; # ... my $exists = exists $hoh{ typo }{ attrib_1 }; # strict can't hear you screa +m... # ... life goes on my $number_of_entities = keys %hoh; # BONK!
The count in $number_of_entities is off by 1 (at least), because now it includes the bogus entity 'typo'.

Or consider this one:

my @big_in_assoc_1 = grep keys %{$assocs_1{$_}} > 25, keys %assocs_2; # ... tics later my @relative_complement = grep !exists %assocs_1{$_}, keys %assocs_2; + # OUCH!
The first line above collects all the entities from table %assocs_2 that have more than 25 attributes in table %assocs_1, but in the process potentially autovivifies any number of empty hashes in %assocs_1 (namely those corresponding to entities in %assocs_2 that were not originally in %assocs_1). So @relative_complement above is always empty.

Of course, to the hardened Perl programmer, the lines above are plainly foolish, just asking for it. But to the greenhorn they look pretty reasonable, cool even.

Those were days of interminable debugging, of endless wading through the muck with DB's s. We went nuts, and some of us never recovered. My buddy... my buddy... Last time I heard of him he was programming Python somewhere out West.

Those of us who pulled through have had to learn to live with the Ugly. Gone are the carefree days when keys was my trusted friend, the only tool I needed to find the size of a table:

my $number_of_entities = grep defined $hoh{$_}, keys %hoh;
Now I know the insidious treachery keys is capable of:
my @big_in_assoc_1 = grep $assocs_1{$_} && keys %{$assocs_1{$_}} > 25, keys %assocs_2;
There's more, but these painful memories are mostly blocked.

the lowliest monk