perldoc perlop says:

Binary "%" computes the modulus of two numbers. Given integer operands "$a" and "$b": If "$b" is positive, then "$a % $b" is "$a" minus the largest multiple of "$b" that is not greater than "$a". If "$b" is negative, then "$a % $b" is "$a" minus the smallest multiple of "$b" that is not less than "$a" (i.e. the result will be less than or equal to zero). Note than when "use integer" is in scope, "%" gives you direct access to the modulus operator as implemented by your C compiler. This operator is not as well defined for negative operands, but it will execute faster.

Note the part about "integer operands". I suspect that the arguments got truncated to integers, leading to the cases "4 % 0" and "4 % 2". One can readily imply that Perl does not define % for non-integer arguments. If you need that, you need to write your own implementation.