Update: I thought now about a tiny test for this:
perl -e '$a=10; eval { $SIG{__WARN__} = sub {  }; $a->isa('foo')}; print "Eval: ", $@, "\n"'
or
perl -e '$a=10; eval { $SIG{__WARN__} = undef; $a->isa('foo')}; print "Eval: ", $@, "\n"'

Both have the warning still in $@. I can't say much about a tied STDERR but it looks pretty much that this isn't working. One another idea I had was to localise the STDERR typeglob inside eval:

perl -e '$a=10; eval { local *STDERR; print STDERR "Test\n"; $a->isa('foo'); }; print $@;'

While this still sets the warning inside $@ the message to STDERR isn't printed. It's worth a try.