Accent Characters and Text::CSV

Anonymous Monk has asked for the wisdom of the Perl Monks concerning the following question:

I have a CSV file which contains a long list of wines and spirits. I am using Text::CSV_XS to load the data into a database. However my problems arise when the wines use foreign characters which fall outside the allowed range.

Taken from the docs

"Allowable characters within a CSV field include 0x09 (tab) and the inclusive range of 0x20 (space) through 0x7E (tilde). In binary mode all characters are accepted, at least in quoted fields"

I assume this is the problem anyway, anyone got any ideas how to get around this? Is there a hack I can make which will allow me to use this module? SHould I replace all dodgy foreign accents etc, before processing...? At least the file would work then? Or is there another CSV module that anyone recommends? I've used quite a few now, and keep changing for various different reasons... I really liked this one until now.

Thanks guys
Tom

Comment on Accent Characters and Text::CSV_XS

Replies are listed 'Best First'.
Re: Accent Characters and Text::CSV_XS by borisz (Canon) on Jan 21, 2005 at 12:44 UTC
Have you tried to open Text::CSV_XS in binary mode? `Text::CSV_XS->new(binary => 1,...);` [download] Another good module is Text::xSV. Boris	[reply] [d/l]
Re^2: Accent Characters and Text::CSV_XS by jZed (Prior) on Jan 21, 2005 at 17:55 UTC
Yes, borisz is correct, if you start Text::CSV_XS in binary mode, all of the characters will be accepted. Minor caveat - the new() method takes a hashref, not a hash, so the correct way is: `my $csv = Text::CSV_XS->new( {binary=>1} );` [download] The next release of Text::CSV_XS will provide an option for defaulting for this behaviour rather than having to declare it on a script-by-script basis.	[reply] [d/l]
Re^3: Accent Characters and Text::CSV_XS by existem (Sexton) on Jan 21, 2005 at 18:22 UTC
thanks for the help, I will try that out. cheers, Tom	[reply]
Re: Accent Characters and Text::CSV_XS by existem (Sexton) on Jan 21, 2005 at 12:22 UTC
sorry that last post was by me, I forgot to login (again! doh)	[reply]


Syntactic Confectionery Delight
	PerlMonks

Accent Characters and Text::CSV_XS