I would like to know what it is about my pattern that creates all those undefs

The match operator returns the value captured by each capture. The following has two captures:

/(?: (\d+) | (?:"(.+?)")  ) /gx
     ^           ^
     |           |
[download]

so two values are returned for each match. Given the pattern, one will always be undef since one will always be outside the path that matched.

It's not evident from your code that it would be a problem to only return one value because you treat both values equally. In real life, you almost always want to treat the two kinds of matches differently. For example,

sub dequote {
    my ($s) = @_;
    $s =~ s/\\(.)/$1/g;
    return $s;
}

push @matches, defined($1) ? $1 : dequote($2)
   while /(?: (\d+) | (?:"(.+?)")  ) /gx;
[download]

If the pattern only returned one value, you wouldn't be able to tell which part of the pattern matched, so you couldn't take decisions based on that (such as whether to call dequote or not).