I haven't used it, but App::Stacktrace claims to do just that.

Looking at the source, it basically seems to automate gdb to step through things, and it seems to only know about Perl up to 5.14, so some manuall massaging of the code might be needed.

There is Debugging Perl scripts which use fork(), and there, Debug::Fork::Tmux is mentioned, which doesn't use Xterms on the client but Tmux sessions on the server.