http://qs321.pair.com?node_id=11115280

Here is a collection of subroutines for converting between:

I really needed one just recently after my older implementation broke because of unicode content and Data::Dumper's obsession with escaping unicode. And here is what I have whipped up for my own use and anyone else's after posting Convert JSON to Perl and back with unicode and getting pointers from haukex, kcott, an anonymous monk and Corion who solved (hopefully for eternity) how to make Data::Dumper unicode escaping optional,(see Corion's answer Re: Convert JSON to Perl and back with unicode).

Because I don't leave a challenge unchallenged here is the code, in the hope to be released as a module with your comments and suggestions.

package Data::Roundtrip; use 5.006; use strict; use warnings; use Encode qw/encode_utf8 decode_utf8/; use JSON qw/decode_json encode_json/; use Unicode::Escape; use YAML; use Sub::Override; use Data::Dumper qw/Dumper/; use Exporter qw(import); our @EXPORT = qw( _read_from_file _write_to_file _read_from_filehandle _write_to_filehandle perl2json json2perl perl2yaml yaml2perl perl2dump dump2perl ); sub _read_from_file { my $infile = $_[0]; my $FH; if( ! open $FH, '<:encoding(UTF-8)', $infile ){ warn "failed to open file '$infile' for reading, $!"; return undef; } my $contents = _read_from_filehandle($FH); close $FH; return $contents } sub _write_to_file { my $outfile = $_[0]; my $contents = $_[1]; my $FH; if( ! open $FH, '>:encoding(UTF-8)', $outfile ){ warn "failed to open file '$outfile' for reading, $!"; return undef } if( ! _write_to_filehandle($FH, $contents) ){ warn "error, call to + ".'_write_to_filehandle()'." has failed"; close $FH; return undef } close $FH; return $contents } sub _read_from_filehandle { my $FH = $_[0]; # you should open INFH as '<:encoding(UTF-8)' # or if it is STDIN, do binmode STDIN , ':encoding(UTF-8)'; return do { local $/; <$FH> } } sub _write_to_filehandle { my $FH = $_[0]; my $contents = $_[1]; # you should open $OUTFH as >:encoding(UTF-8)' # or if it is STDOUT, do binmode STDOUT , ':encoding(UTF-8)'; print $FH $contents; return 1; } sub perl2json { my $pv = $_[0]; my $params = defined($_[1]) ? $_[1] : {}; my $pretty_printing = exists($params->{'pretty'}) && defined($para +ms->{'pretty'}) ? $params->{'pretty'} : 0 ; my $escape_unicode = exists($params->{'escape-unicode'}) && define +d($params->{'escape-unicode'}) ? $params->{'escape-unicode'} : 0 ; my $json_string; if( $escape_unicode ){ if( $pretty_printing ){ $json_string = JSON->new->utf8(1)->pretty->encode($pv); } else { $json_string = JSON->new->utf8(1)->encode($pv) } $json_string = Unicode::Escape::escape($json_string, 'utf8'); } else { if( $pretty_printing ){ $json_string = JSON->new->utf8(0)->pretty->encode($pv); } else { $json_string = JSON->new->utf8(0)->encode($pv) } } if( ! $json_string ){ warn "perl2json() : error, no json produced +from perl variable"; return undef } return $json_string } sub perl2yaml { my $pv = $_[0]; my $params = defined($_[1]) ? $_[1] : {}; my $pretty_printing = exists($params->{'pretty'}) && defined($para +ms->{'pretty'}) ? $params->{'pretty'} : 0 ; warn "perl2yaml() : pretty-printing is not supported" and $pretty_ +printing=0 if $pretty_printing; my $escape_unicode = exists($params->{'escape-unicode'}) && define +d($params->{'escape-unicode'}) ? $params->{'escape-unicode'} : 0 ; my ($yaml_string, $escaped); if( $escape_unicode ){ if( $pretty_printing ){ #$yaml_string = Data::Format::Pretty::YAML::format_pretty( +$pv); } else { $yaml_string = YAML::Dump($pv) } $yaml_string = Unicode::Escape::escape($yaml_string, 'utf8'); } else { if( $pretty_printing ){ #$yaml_string = Data::Format::Pretty::YAML::format_pretty( +$pv); } else { $yaml_string = YAML::Dump($pv) } } if( ! $yaml_string ){ warn "perl2yaml() : error, no yaml produced +from perl variable"; return undef } # return Encode::decode_utf8($yaml_string) return $yaml_string } sub yaml2perl { my $yaml_string = $_[0]; #my $params = defined($_[1]) ? $_[1] : {}; my $pv = YAML::Load($yaml_string); if( ! $pv ){ warn "yaml2perl() : error, call to YAML::Load() has f +ailed"; return undef } return $pv } sub json2perl { my $json_string = $_[0]; #my $params = defined($_[1]) ? $_[1] : {}; my $pv = JSON::decode_json(Encode::encode_utf8($json_string)); if( ! defined $pv ){ warn "json2perl() : error, call to json2perl +() has failed"; return undef } return $pv; } sub yaml2json { my $yaml_string = $_[0]; my $params = defined($_[1]) ? $_[1] : {}; my $pv = yaml2perl($yaml_string, $params); if( ! $pv ){ warn "yaml2json() : error, call to yaml2perl() has fa +iled"; return undef } my $json = perl2json($pv, $params); if( ! $json ){ warn "yaml2json() : error, call to perl2json() has +failed"; return undef } return $json } sub json2yaml { my $json_string = $_[0]; my $params = defined($_[1]) ? $_[1] : {}; my $pv = json2perl($json_string, $params); if( ! defined $pv ){ warn "json2yaml() : error, call to json2perl +() has failed"; return undef } my $yaml_string = perl2yaml($pv, $params); if( ! defined $yaml_string ){ warn "json2yaml() : error, call to +perl2yaml() has failed"; return undef } return $yaml_string } # this bypasses Data::Dumper's obsession with escaping # non-ascii characters by redefining qquote() sub # The redefinition code is by [Corion] @ Perlmonks and cpan # see https://perlmonks.org/?node_id=11115271 # sub perl2dump { my $pv = $_[0]; my $params = defined($_[1]) ? $_[1] : {}; local $Data::Dumper::Terse = exists($params->{'terse'}) && defined +($params->{'terse'}) ? $params->{'terse'} : 0 ; local $Data::Dumper::Indent = exists($params->{'indent'}) && defin +ed($params->{'indent'}) ? $params->{'indent'} : 0 ; my $ret; if( exists($params->{'dont-bloody-escape-unicode'}) && defined($pa +rams->{'dont-bloody-escape-unicode'}) && ($params->{'dont-bloody-escape-unicode'}==1) ){ local $Data::Dumper::Useperl = 1; local $Data::Dumper::Useqq='utf8'; my $override = Sub::Override->new( 'Data::Dumper::qquote' => \& _qquote_redefinition_by_Corio +n ); $ret = Dumper($pv); # restore the overriden sub $override->restore; } else { $ret = Dumper($pv) } return $ret } sub dump2perl { my $dump_string = $_[0]; #my $params = defined($_[1]) ? $_[1] : {}; $dump_string =~ s/^\$VAR1\s*=\s*//g; my $pv = eval($dump_string); if( $@ || ! defined $pv ){ warn "error, failed to eval() input str +ing alledgedly a perl variable: $@"; return undef } return $pv } # Below code is by [Corion] @ Perlmonks and cpan # see https://perlmonks.org/?node_id=11115271 # it's for redefining Data::Dumper::qquote # (it must be accompanied by # $Data::Dumper::Useperl = 1; # $Data::Dumper::Useqq='utf8'; sub _qquote_redefinition_by_Corion { local($_) = shift; s/([\\\"\@\$])/\\$1/g; return qq("$_") unless /[[:^print:]]/; # fast exit if only printabl +es # Here, there is at least one non-printable to output. First, trans +late the # escapes. s/([\a\b\t\n\f\r\e])/$Data::Dumper::esc{$1}/g; # no need for 3 digits in escape for octals not followed by a digit. s/($Data::Dumper::low_controls)(?!\d)/'\\'.sprintf('%o',ord($1))/eg; # But otherwise use 3 digits s/($Data::Dumper::low_controls)/'\\'.sprintf('%03o',ord($1))/eg; # all but last branch below not supported --BEHAVIOR SUBJECT TO CH +ANGE-- my $high = shift || ""; if ($high eq "iso8859") { # Doesn't escape the Latin1 printables if ($Data::Dumper::IS_ASCII) { s/([\200-\240])/'\\'.sprintf('%o',ord($1))/eg; } elsif ($] ge 5.007_003) { my $high_control = utf8::unicode_to_native(0x9F); s/$high_control/sprintf('\\%o',ord($1))/eg; } } elsif ($high eq "utf8") { # Some discussion of what to do here is in # https://rt.perl.org/Ticket/Display.html?id=113088 # use utf8; # $str =~ s/([^\040-\176])/sprintf "\\x{%04x}", ord($1)/ge; } elsif ($high eq "8bit") { # leave it as it is } else { s/([[:^ascii:]])/'\\'.sprintf('%03o',ord($1))/eg; #s/([^\040-\176])/sprintf "\\x{%04x}", ord($1)/ge; } return qq("$_"); } =head1 NAME Data::Roundtrip - convert between Perl data structures, YAML and JSON +with unicode support (I believe ...) =head1 VERSION Version 0.01 =cut our $VERSION = '0.01'; =head1 SYNOPSIS Quick summary of what the module does. Perhaps a little code snippet. use Data::Roundtrip; my $foo = Data::Roundtrip->new(); ... =head1 EXPORT perl2json json2perl perl2yaml yaml2perl perl2dump dump2perl _read_from_file _write_to_file _read_from_filehandle _write_to_filehandle =head1 SUBROUTINES/METHODS =head2 perl2json($perlvar, $optional_paramshashref) Given an input C<$perlvar> (which can be a simple scalar or a nested data structure, but not an object), it will return the equivalent JSON string. In C<$optional_paramshashref> one can specify whether to escape unicode with C<'escape-unicode'=>1> and/or prettify the returned result with C<'pretty'=>1>. The output can fed to L<Data::Roundtrip::json2perl> for getting the Perl variable back. Returns the JSON string on success or C<undef> on failure. =head2 json2perl($jsonstring) Given an input C<$jsonstring> as a string, it will return the equivalent Perl data structure using C<JSON::decode_json(Encode::encode_utf8($jsonstring))>. Returns the Perl data structure on success or C<undef> on failure. =head2 perl2yaml($perlvar, $optional_paramshashref) Given an input C<$perlvar> (which can be a simple scalar or a nested data structure, but not an object), it will return the equivalent YAML string. In C<$optional_paramshashref> one can specify whether to escape unicode with C<'escape-unicode'=>1>. Prettify is not supported yet. The output can fed to L<Data::Roundtrip::yaml2perl> for getting the Perl variable back. Returns the YAML string on success or C<undef> on failure. =head2 yaml2perl($yamlstring) Given an input C<$yamlstring> as a string, it will return the equivalent Perl data structure using C<YAML::Load($yamlstring)> Returns the Perl data structure on success or C<undef> on failure. =head2 perl2dump($perlvar, $optional_paramshashref) Given an input C<$perlvar> (which can be a simple scalar or a nested data structure, but not an object), it will return the equivalent string (via L<Data::Dumper>). In C<$optional_paramshashref> one can specify whether to NOT escape unicode with C<'dont-bloody-escape-unicode'=>1>, and/or use terse output with C<'terse'=>1> and remove all the incessant indentation C<indent'=>1> which unfortunately goes to the other extreme of producing a space-less output, not fit for human consumption. The output can fed to L<Data::Roundtrip::dump2perl> for getting the Perl variable back. Returns the string representation of the input perl variable on success or C<undef> on failure. =head2 json2perl($jsonstring) Given an input C<$jsonstring> as a string, it will return the equivalent Perl data structure using C<JSON::decode_json(Encode::encode_utf8($jsonstring))>. Returns the Perl data structure on success or C<undef> on failure. In C<$optional_paramshashref> one can specify whether to escape unicode with C<'escape-unicode'=>1> and/or prettify the returned result with C<'pretty'=>1>. Returns the yaml string on success or C<undef> on failure. =head2 json2yaml($jsonstring, $optional_paramshashref) Given an input JSON string C<$jsonstring>, it will return the equivalent YAML string L<YAML> by first converting JSON to a Perl variable and then converting that variable to YAML using L<Data::Roundtrip::perl2yaml()> +. All the parameters supported by L<Data::Roundtrip::perl2yaml()> are accepted. Returns the YAML string on success or C<undef> on failure. =head2 yaml2json($yamlstring, $optional_paramshashref) Given an input YAML string C<$yamlstring>, it will return the equivalent YAML string L<YAML> by first converting YAML to a Perl variable and then converting that variable to JSON using L<Data::Roundtrip::perl2json()> +. All the parameters supported by L<Data::Roundtrip::perl2json()> are accepted. Returns the JSON string on success or C<undef> on failure. =head1 AUTHOR Andreas Hadjiprocopis, C<< <bliako at cpan.org> >> =head1 BUGS Please report any bugs or feature requests to C<bug-data-roundtrip at +rt.cpan.org>, or through the web interface at L<https://rt.cpan.org/NoAuth/ReportBug.html?Queue +=Data-Roundtrip>. I will be notified, and then you'll automatically be notified of progress on your bug as I make changes. =head1 SUPPORT You can find documentation for this module with the perldoc command. perldoc Data::Roundtrip You can also look for information at: =over 4 =item * RT: CPAN's request tracker (report bugs here) L<https://rt.cpan.org/NoAuth/Bugs.html?Dist=Data-Roundtrip> =item * AnnoCPAN: Annotated CPAN documentation L<http://annocpan.org/dist/Data-Roundtrip> =item * CPAN Ratings L<https://cpanratings.perl.org/d/Data-Roundtrip> =item * Search CPAN L<https://metacpan.org/release/Data-Roundtrip> =back =head1 ACKNOWLEDGEMENTS =head1 LICENSE AND COPYRIGHT This software, EXCEPT the portion created by [Corion] @ Perlmonks, is Copyright (c) 2020 by Andreas Hadjiprocopis. This is free software, licensed under: The Artistic License 2.0 (GPL Compatible) =cut 1; # End of Data::Roundtrip

And here is a test script which demonstrates usage and tests unicoded content:

#!perl -T
use 5.006;
use strict;
use warnings;

use utf8;
binmode STDERR, ':encoding(UTF-8)';
binmode STDOUT, ':encoding(UTF-8)';
binmode STDIN,  ':encoding(UTF-8)';
# to avoid wide character in TAP output
# do this before loading Test* modules
use open ':std', ':encoding(utf8)';

use Test::More;
#use Test::Deep;

my $num_tests = 0;

use Data::Roundtrip;

use Data::Dumper qw/Dumper/;

my $abc = "abc-αβγ";
my $xyz = "χψζ-xyz";

my $json_string = <<EOS;
{"$abc":"$xyz"}
EOS
$json_string =~ s/\s*$//;

my $yaml_string = <<EOS;
---
$abc: $xyz
EOS
#$yaml_string =~ s/\s*$//;

my $perl_var = {$abc => $xyz};

# perl2json
my $result = Data::Roundtrip::perl2json($perl_var);
ok(defined $result, "perl2json() called."); $num_tests++;
ok($result eq $json_string, "perl2json() checked (got: '$result', expected: '$json_string')."); $num_tests++;

# json2perl
$result = Data::Roundtrip::json2perl($json_string);
ok(defined $result, "json2perl() called."); $num_tests++;
for (keys %$result){
	ok(exists $perl_var->{$_}, "json2perl() key exists."); $num_tests++;
	ok($perl_var->{$_} eq $result->{$_}, "json2perl() values are the same."); $num_tests++;
}
for (keys %$perl_var){
	ok(exists $result->{$_}, "json2perl() key exists (other way round)."); $num_tests++;
}
# this fails:
#cmp_deeply($perl_var, $result, "json2perl() checked (got: '".Dumper($result)."', expected: ".Dumper($perl_var).")."); $num_tests++;

# perl2yaml
$result = Data::Roundtrip::perl2yaml($perl_var);
ok(defined $result, "perl2yaml() called."); $num_tests++;
ok($result eq $yaml_string, "perl2yaml() checked (got: '$result', expected: '$yaml_string')."); $num_tests++;

# yaml2perl
$result = Data::Roundtrip::yaml2perl($yaml_string);
ok(defined $result, "yaml2perl() called."); $num_tests++;
for (keys %$result){
	ok(exists $perl_var->{$_}, "yaml2perl() key exists."); $num_tests++;
	ok($perl_var->{$_} eq $result->{$_}, "yaml2perl() values are the same."); $num_tests++;
}
for (keys %$perl_var){
	ok(exists $result->{$_}, "yaml2perl() key exists (other way round)."); $num_tests++;
}

# yaml2json
$result = Data::Roundtrip::yaml2json($yaml_string);
ok(defined $result, "yaml2json() called."); $num_tests++;
ok($result eq $json_string, "perl2yaml() checked (got: '$result', expected: '$json_string')."); $num_tests++;

# json2yaml
$result = Data::Roundtrip::json2yaml($json_string);
ok(defined $result, "json2yaml() called."); $num_tests++;
ok($result eq $yaml_string, "perl2yaml() checked (got: '$result', expected: '$yaml_string')."); $num_tests++;

# perl2dump and dump2perl with unicode quoting (default Data::Dumper behaviour)
my $adump = Data::Roundtrip::perl2dump($perl_var, {'terse'=>1});
ok(defined $adump, "perl2dump() called."); $num_tests++;
ok($adump=~/\\x\{3b1\}/, "perl2dump() unicode quoted."); $num_tests++;
# dump2perl
$result = Data::Roundtrip::dump2perl($adump);
ok(defined $result, "dump2perl() called."); $num_tests++;
for (keys %$result){
	ok(exists $perl_var->{$_}, "perl2dump() and dump2perl() key exists."); $num_tests++;
	ok($perl_var->{$_} eq $result->{$_}, "perl2dump() and dump2perl() values are the same."); $num_tests++;
}
for (keys %$perl_var){
	ok(exists $result->{$_}, "perl2dump() and dump2perl() key exists (other way round)."); $num_tests++;
}

# perl2dump and dump2perl WITHOUT unicode quoting
$adump = Data::Roundtrip::perl2dump($perl_var, {'terse'=>1, 'dont-bloody-escape-unicode'=>1});
ok(defined $adump, "perl2dump() called."); $num_tests++;
ok($adump!~/\\x\{3b1\}/, "perl2dump() unicode not quoted."); $num_tests++;
# dump2perl
$result = Data::Roundtrip::dump2perl($adump);
ok(defined $result, "dump2perl() called."); $num_tests++;
for (keys %$result){
	ok(exists $perl_var->{$_}, "perl2dump() and dump2perl() key exists."); $num_tests++;
	ok($perl_var->{$_} eq $result->{$_}, "perl2dump() and dump2perl() values are the same."); $num_tests++;
}
for (keys %$perl_var){
	ok(exists $result->{$_}, "perl2dump() and dump2perl() key exists (other way round)."); $num_tests++;
}

done_testing($num_tests);

bw, bliako

Replies are listed 'Best First'.
Re: RFC: Perl<->JSON<->YAML<->Dumper : roundtripping and possibly with unicode
by kcott (Archbishop) on Apr 09, 2020 at 18:00 UTC

    G'day bliako,

    As requested, here's some comments and suggestions.

    Naming

    Subroutine names with a leading underscore tend to suggest private methods or implementation details; these are generally for internal use and not for users of module, i.e. not part of the interface — _qquote_redefinition_by_Corion() is a part of the implementation and the leading underscore is appropriate. The other four subroutines (_read_from_* and _write_to_*) are, in my opinion, part of the interface — consider removing the leading underscore from these.

    Exporting

    I wouldn't export those ten subroutines by default (i.e. via @EXPORT); in fact, I wouldn't export anything by default, instead using @EXPORT_OK as recommended in "Exporter: Selecting What to Export".

    In addition to using @EXPORT_OK, I would add %EXPORT_TAGS so that users of the module do not have to write long import lists. The following is an example of what can be done; it is not a recommendation of what you should do, although I do think something along these lines is a good idea.

    use Exporter 'import'; our (@EXPORT_OK, %EXPORT_TAGS); BEGIN { my @file = qw{read_from_file write_to_file}; my @fh = qw{read_from_filehandle write_to_filehandle}; my @io = (@file, @fh); my @json = qw{perl2json json2perl}; my @yaml = qw{perl2yaml yaml2perl}; my @dump = qw{perl2dump dump2perl}; my @all = (@io, @json, @yaml, @dump); @EXPORT_OK = @all; %EXPORT_TAGS = ( file => [@file], fh => [@fh], io => [@io], json => [@json], yaml => [@yaml], dump => [@dump], all => [@all], ); }

    Layout

    Some of your code layout could be improved. Lines like this one are not easy to read:

    if( ! _write_to_filehandle($FH, $contents) ){ warn "error, call to ".' +_write_to_filehandle()'." has failed"; close $FH; return undef }

    That's 134 characters long; plus there's leading whitespace for indentation. This would be much more readable, at least in my opinion, as:

    if( ! _write_to_filehandle($FH, $contents) ) { warn "error, call to ".'_write_to_filehandle()'." has failed"; close $FH; return undef; }

    — Ken

      Thank you kcott for your comments. I will adopt "Exporting" 100%. For the last one, "Layout", here is the logic to this madness: I try to put all "boring" code in one long line, like : if error then print blah blah blah and return error A long line for me means that all is lost, sub is exiting. If it's a warning I always make it multiline like you suggest. For your suggestion "Naming", I will just wait and see which of those will make it public or not.

      thank you

Re: RFC: Perl<->JSON<->YAML<->Dumper : roundtripping and possibly with unicode
by jwkrahn (Abbot) on Apr 09, 2020 at 21:25 UTC
    $ podchecker 11115280.pl *** WARNING: (section) in 'Data::Roundtrip::perl2yaml()' deprecated at + line 346 in file 11115280.pl *** WARNING: (section) in 'Data::Roundtrip::perl2yaml()' deprecated at + line 347 in file 11115280.pl *** WARNING: (section) in 'Data::Roundtrip::perl2json()' deprecated at + line 357 in file 11115280.pl *** WARNING: (section) in 'Data::Roundtrip::perl2json()' deprecated at + line 358 in file 11115280.pl *** WARNING: empty section in previous paragraph at line 410 in file 1 +1115280.pl 11115280.pl pod syntax OK.
    sub _write_to_file { ^^^^^ my $outfile = $_[0]; my $contents = $_[1]; my $FH; if( ! open $FH, '>:encoding(UTF-8)', $outfile ){ ^^ warn "failed to open file '$outfile' for reading, $!"; ^^^^^^^

      Thanks for this check. I have fixed the last one. And also the last Warning (line 410), it was an empty section. But I don't understand the other warnings. (plus my podchecker for perl v5.28.2 does not show these warnings). this is the context fro one of those:

      =head2 perl2yaml($perlvar, $optional_paramshashref) Given an input C<$perlvar> (which can be a simple scalar or a nested data structure, but not an object), it will return the equivalent YAML string. In C<$optional_paramshashref> one can specify whether to escape unicode with C<'escape-unicode'=E<gt>1>. Prettify is not supported yet. The output can fed to L<Data::Roundtrip::yaml2perl> for getting the Perl variable back.

      also, do you have an opinion about having function names with params? I don't like it when I read a manual and it mentions only sub names. So I made it very explicit what it takes in

        I didn't run a POD checker; however, by visual inspection, I see four instances like

        C<'escape-unicode'=>1>

        Given there are also four, very similar warnings, that you had difficulties identifying, I'd hazard a guess that they're the source.

        I see in your post (to which I'm replying) you've changed one of those to

        C<'escape-unicode'=E<gt>1>

        You may find it easier, and more readable, to use

        C<< 'escape-unicode'=>1 >>

        That's explained in "perlpod: Formatting Codes" (you'll need to scroll down a fair way; it's past all the initial dot points in that section).

        I'd also recommend you add this as you first POD command paragraph:

        =encoding utf8

        That will cover any Greek, or other non-ASCII, examples you may add. See "perlpod: Command Paragraph: =encoding" for details.

        "... do you have an opinion about having function names with params? I don't like it when I read a manual and it mentions only sub names."

        Yes, I agree that the arguments should be documented; in addition, the return value should also be described. I would typically write the example you cite something along these lines:

        =head2 C<perl2yaml> my $yaml = perl2yaml($perlvar, $optional_paramshashref); The C<perl2yaml> function does ... Arguments: =over 4 =item * C<$perlvar> ... C<$perlvar> description ... =item * C<$optional_paramshashref> ... C<$optional_paramshashref> description ... =back Return value: =over 4 =item * C<$yaml> ... C<$yaml> description ... =back

        Although that looks like a lot of typing for every function, I'd generally set up a template for all of the formatting: that only leaves specifics to be entered. And, where you have very similar documentation, a copy, paste, and substitution can do most of the work; for instance, you could make three copies of the above then s/yaml/json/g in one copy and s/yaml/dump/g in another, and most of your work is done (the three xxxx2perl functions could be handled similarly).

        — Ken

Re: RFC: Perl<->JSON<->YAML<->Dumper : roundtripping and possibly with unicode (Data::Dumper::AutoEncode)
by 1nickt (Canon) on Apr 10, 2020 at 15:29 UTC

    Hi bliako,

    Are you familiar with Data::Dumper::AutoEncode?


    use strict; use warnings; use feature 'say';
    use utf8;
    use Data::Dumper;
    use Data::Dumper::AutoEncode;
    
    my $data = {
        русский  => 'доверяй, но проверяй',
        i中文    => '也許你的生活很有趣',
        Ελληνικά => 'ἓν οἶδα ὅτι οὐδὲν οἶδα',
    };
    
    say Dumper $data;
    say eDumper $data;
    
    __END__
    


    Output:


    $ perl bliako.pl
    
    $VAR1 = {
              "\x{440}\x{443}\x{441}\x{441}\x{43a}\x{438}\x{439}" => "\x{434}\x{43e}\x{432}\x{435}\x{440}\x{44f}\x{439}, \x{43d}\x{43e} \x{43f}\x{440}\x{43e}\x{432}\x{435}\x{440}\x{44f}\x{439}",
              "i\x{4e2d}\x{6587}" => "\x{4e5f}\x{8a31}\x{4f60}\x{7684}\x{751f}\x{6d3b}\x{5f88}\x{6709}\x{8da3}",
              "\x{395}\x{3bb}\x{3bb}\x{3b7}\x{3bd}\x{3b9}\x{3ba}\x{3ac}" => "\x{1f13}\x{3bd} \x{3bf}\x{1f36}\x{3b4}\x{3b1} \x{1f45}\x{3c4}\x{3b9} \x{3bf}\x{1f50}\x{3b4}\x{1f72}\x{3bd} \x{3bf}\x{1f36}\x{3b4}\x{3b1}"
            };
    
    $VAR1 = {
              'русский' => 'доверяй, но проверяй',
              'Ελληνικά' => 'ἓν οἶδα ὅτι οὐδὲν οἶδα',
              'i中文' => '也許你的生活很有趣'
            };
    


    Hope this helps!


    The way forward always starts with a minimal test.

      Thanks 1nickt for the pointer. I may replace the Dumper part with this then!

Published: Perl<->JSON<->YAML<->Dumper : roundtripping and possibly with unicode
by bliako (Monsignor) on Apr 14, 2020 at 07:47 UTC

    I have pushed to PAUSE/CPAN the above as a new package: Data::Roundtrip . I hope it is helpful.

    A big thank you to all who helped with their comments, in this current thread, in this one Convert JSON to Perl and back with unicode and in the CB. If you have more comments, suggestions, feature requests please let me know and I will be happy to incorporate them (reasonable constraints apply).

    bw, bliako

      Don't do this, specify your files, they also don't need the '.pl' file extension. Windows complains `ls` is not recognized as an internal or external command, operable program or batch file. Better just to specify rather than use ls, back-ticks.

      Similarly, why are you setting INSTALL_BASE => "$ENV{HOME}/usr"?

      Lastly, the repo information could be added as I mentioned before, in reference to rt://132244.

        thanks, all three are ommissions or settings that need to change for production, I will see to fixing those.