It was much slower accessing nodes
You seem to have missed "at creating the same Perl-land data structure." Visiting every node was included in the benchmark.
A task using SOAP::Lite that took two minute was sped up to 15 seconds when I replaced Expat with XML::LibXML as its parser. I kept the change mininal: XML::LibXML was used to create the same data structure that Expat was used to create.
Honestly, I haven't tried XML::LibXML::SAX and maybe it is way faster than XML::Parser.
As I mentioned, XML::Parser is by far the fastest existing backend for XML::Simple. I'm pretty sure I've posted benchmarks on PerlMonks.