You don't mention whether your application runs under CGI or a persistent framework such as mod_perl, FastCGI or PersistentPerl.
If you care about performance, I assume you're avoiding CGI so you can reuse database connections and other things that take some time to initialise. Template Toolkit objects are such things: calling Template->new() for each request will make your application run more slowly.
Template Toolkit is used on plenty of high volume Web sites: it's certainly possible to have it run quickly. Unless your application is very simple, TT shouldn't take up 60% of the application's run time.