without breaking anything, is that possible?

No.

It is impossible to change the internals in any important way and keep XS code working.

Several years ago, Nicholas Clark (a perl's internals guru) tried porting perl 5 to parrot (the Perl 6 runtime), and IIRC, he didn't get too far because of XS. See Ponie has been put out to pasture.