I agree with the suggestion above of using double indirection to solve this problem. For one, it's easy to implement and second it's probably a lot more efficient than finding all references in use and resetting them.

AFAIK you can automate the dereferencing process (at the XS side) using typemaps, so you won't have to write any additional XS or perl code to implement this strategy for a straight C++ to perl mapping.