Keep in mind that site_perl is not a perl install by itself, but hold all the modules that are site dependent. Within the site_perl directory, you should have two more: 5.005 and 5.6.0, which contain those libs. I wouldn't be super concerned about them as they take a grand total of 20M apiece (at least on my system, which has many optional modules installed). Besides which, any module you compiled under 5.005 but not under 5.6 will be used by 5.6 (assuming that option was enabled in configure). So I wouldn't worry about removing 5.005.