I suggest to write a wrapper function, called say, Bar(), that calls your XV function, say, XV_bar, by assigning the return value of XV_bar to a temporary variable:

package MyBar;
...
sub Bar {
  my $val = XV_bar(@_);
  return $val;
}
...
# do the XS imports...




# and your code that calls the bar

# Foo.pl
use strict;
use warnings;
use MyBar;

my $n = 1 + MyBar::Bar(...);
# ^ warning will be generated here, not in your module, Bar.pm.
[download]

If the return value is not numeric, your calling code will correctly produce the warning at the calling code, not your module. Also if your calling code does not use warnings, then it will not print any warning messages at all.