http://qs321.pair.com?node_id=11148364


in reply to Re^2: getting rid of UTF-8
in thread getting rid of UTF-8

The issue with the sample data you posted is that it is entirely ASCII with some some BOMs in it, but from your description it sounded like you could have other Latin-1 (or CP1252 or Latin-9) or UTF-8 characters in it, which you don't show.

What should I do either to upload something to here with example probelmatic stuff

A hex dump of the raw bytes like you showed above is fine. See also my node here.

and/or be able just to brute-force fix it?

Iff your data consists entirely a single-byte encoding like the ones I named above, and the only UTF-8 characters that appear in it are BOMs, then the regex you showed in the root node may be acceptable. However, I very much expect that if there's a BOM, then other UTF-8 characters can be present, and if those are mixed with single-byte-encodings, or you've got double-encoded characters, you'll have a tough time picking that apart. But again, you'd need to show us more representative data.

Edit: Typo fixes.