Following the proposed "RHS decides the op", then when the RHS is an actual number, it could coerce the LHS to a number.

It's the when the RHS is an actual number that's problematic. When do you consider something a number? The fact that I have happened to know that both Mojo::JSON and JSON::Tiny recently adjusted their heuristic for what to consider a number (and at least Mojo::JSON has been around for quite some time before) doesn't make me very optimistic that there is a reasonable way to answer that question in perl 5, precisely because of the automagic coercion / SV-NV duality.