The literal reference is just a value which in your map just happens to be contained in $_. When you reassign 1 to it, it's not changing the reference in any way, you're just putting something else in $_ (although this is where I'm getting fuzzy and agree it's somewhat weird); it's just that $_ just happens to be aliased to some temporary SV* / SVrv that the map is using as its argument list. The assignment isn't changing the referenced value, it's just sticking the new SViv 1 into that SV*.
DB<1> $a = $b = [qw/a b c/]
DB<2> x map { $_ = 1 } $a
0 1
DB<3> x $a
0 1
DB<4> x $b
0 ARRAY(0x7fa8144335b0)
0 'a'
1 'b'
2 'c'
Hopefully someone more conversant in perlguts than I might can explain it better.
The cake is a lie.
The cake is a lie.
The cake is a lie.