I personally believe that the "only" risk associated with it is that you can pass arbitrary code to be executed, so it all depends on whom do you plan to make your script available to and to security settings. Thus

perl -pe ''
[download]

can be considered as an equivalent to cat(1) (well, except for the bazillion cmd line switches, of course!) but if you do

perl -pe '' "rm -rfv $HOME |"
[download]

then you're on your own. However FWIW it just boils down to doing

rm -rfv $HOME
[download]

in the first place. You "just" have to make sure others are not allowed to do the same or similar things, if you use the diamond operator: of course the issue has nothing to do with Perl anymore, but with your system's administration instead.