Thanks for asking this question! At $work many years ago, I faced this same issue and ended up writing my own, half-finished graphics abstraction layer. The layer allows plotting certain types of planar graphs with a GD backend and a separate SVG backend -- the latter is another bespoke module, because there wasn't anything suitable in CPAN back then... Our code sadly isn't suitable for CPAN, because it solves a very narrow problem and we don't have the resources to polish and maintain it as an external product.
It's a bit sad that things haven't moved on and rasterising SVGs in Perl is still on such a shaky ground. We ruled out Image::LibRSVG back in 2008 due to code quality concerns and compilation problems on Windows. I didn't know about GD::SVG, though.