Actually, you do good by reporting it to their ISP. Sure, he's not going to get busted for checking for vulnerabilities on websites...life isn't that easy. But...if you and 30 other people mention that this guy did this, the ISP is definitely going to take note of it, and maybe give the script kiddie (or his parents) a call to find out what's happening.
On the other hand, if you do nothing, he'll get bored of it and either decide playing Half-life is more fun, or start looking at scanners and rootkits. Personally, as a working sysadmin, I'd hope that people would give a little effort and scare him a little before he does something that's going to get him in serious trouble or even jail down the road. If you do nothing, nothing gets better. Complacency is the bane of security, both in securing your own systems and in making the net a better place to be.
Kickstart