4K (or whatever buffer size perl is using) is likely to be a pretty good size for an input buffer, based on lots of experience and tweaking among perl maintainers. It strikes a nice balance between competing resource demands -- a larger or smaller size might improve some things, but hinder others.
Your processing is going to be line-oriented anyway, and perl's internal buffering is already optimized (in C) to deliver lines while managing the underlying block-oriented buffering.
If you try doing the buffering yourself (e.g. using read as suggested in another reply), you'll end up slowing things down, because you have to write your own code to figure out the line boundaries, retain a buffer-final line fragment so that you can append the next buffer to that, and so on. It's not only slower to run, it's slower and harder to code, test and maintain.
If the runtime speed of the standard while (<>) loop in perl is a serious issue for your task, maybe you just need to use C. But then you'll spend even more time for coding, testing and maintenance. It's a question of whose time is more important and expensive: the programmer's, or the cpu's.
