Something like ten years ago, I wrote a RADIUS authentication server in Perl. The RADIUS protocol involves sending blocks of binary data in UDP packets.
My skills being what they were ten years ago, I lived in constant fear that my carefully crafted numbers and strings would be converted from one to the other behind my back. I've sadly forgotten the details of this ordeal, but I've carried the "don't write a binary protocol handler in Perl" scars around with me ever since.
As I think about it now, if I were writing it today, I don't think I'd have much problem. I'd make a RADIUS packet object to abstract away all kinds of headaches. I still would have to study pack and unpack a while before I got it right, but I'd do a lot of testing to help guide my way.