But if used on the right side this function deletes the variable from the symbol table

No, that's not quite correct. An uninitalized variable such as my $x; is initially undef, and saying either $x=undef; or undef $x; will return it back to this state of being "undefined" (holding no value). This has nothing to do with the symbol table, it remains unaffected by the value assigned to the variable; and undef is just a special value that a variable can hold. (Update: I should be a bit more specific and say "scalar" instead of "variable". Arrays and hashes are different from scalars.)

undef is indeed a bit special in the left side of a list assignment, in that it acts as a placeholder there, as you showed.

Update 2: I responded to your edit here.

Update 3: Responding to your further edits:

I think undef is not a value, as many here assume, but a function

No, in a sense it is both.