When dealing with web security you can never be too paranoid (that's what THEY want you to think; but perhaps I've said too much . . .).
That said, it really depends on what kind of files they are and what you're doing with them. Image files of some sort (JPGs, PNGs), probably not an issue. HTML that could possibly contain malicious Javascript that you're going to display to other users, that you might go over more carefully.
Update: Oop, good point about the PNG buffer overflow; good thing I weaseled and said "probably". :)
