What have I got wrong?

You mistranscribed that \d*+.
It should be \d++.
(I checked back through all of my notes and presentations, and its definitely shown as \d++ in them.)

The nested match goes wrong when you use \d*+, because when it encounters a nested  < the \d*+ successfully matches zero times.

And, because the \d*+ | (?&LIST) is itself in a non-backtracking loop: (\d*+ | (?&LIST) )*+, when the zero-length submatch causes the main match to fail, the regex engine can't backtrack into the alternative and try the (?&LIST) instead.

So the original match from the start of the string fails, and the regex engine skips down the string, trying again and again, until it finds the nested sublist, which it is able to match.

Incidentally, you could have watched this happen live and in colour via the Regexp::Debugger module. Just download it from CPAN and add it in front of your regex:

use Regexp::Debugger;
my $re=qr{(?x) (?&LIST)
        (?(DEFINE)
            (?<LIST> < (?&ITEM)  (?: , (?&ITEM))*+ > )
            (?<ITEM> \d*+ | (?&LIST) )
        )
        };
[download]