Should one use the same layers in the same order for both input and output?

They are processed from the file handle out when reading, and in the opposite direction when writing.

Also, do you know why it doesn't work with the open pragma?

Maybe it does the equivalent of binmode, and binmode doesn't remove the existing layers. (:raw simply ends up disabling the crlf layer, then :crlf reenables the existing layer rather than adding a new layer.)