Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl: the Markov chain saw
 
PerlMonks  

Writing multilingual perl applications?

by skx (Parson)
on Mar 19, 2003 at 20:44 UTC ( [id://244452]=perlquestion: print w/replies, xml ) Need Help??

skx has asked for the wisdom of the Perl Monks concerning the following question:

 I've written a reasonably large application in perl, and now I'm looking to internalionalise it.

 Are there any modules that will help? Or gotchas that I should avoid?

 Initially I was thinking I could have something like this:

$TEXT_STRINGS{ 'not_found_uk' } = "File not found"; $TEXT_STRINGS{ 'not_found_de' } = "File nichte founden";

 But now I realise that I have to be slightly more complex and take into account the placement of text I insert - I want to be able to say "Foo is not valid", which might result in "not valide Foo" in some languages.

 A similar problem exists when trying to deal with plurals.

 A search of CPAN didn't result in anything obvious, so I'd appreciate the advice of any wise monks in the area.

Steve
---
steve.org.uk

Replies are listed 'Best First'.
Re: Writing multilingual perl applications?
by dakkar (Hermit) on Mar 19, 2003 at 21:04 UTC

    Watch out for:

    • Encoding/charset problems. In practice, don't use a Perl before 5.8, and keep all your strings in a utf-8-encoded file. Be prepared to transcode them (see Encode) for the display, though.
    • Plural forms. Just because some languages have two forms, don't assume every other has too. There are languages with a different form for "2 thing" and "more than 2 things", or different forms depending on $x%10. And 0 is singular in some languages and plural in others...

    I'd advise you to look into the modifications made to gettext and family by the KDE team. A couple of friends of mine are on the italian translation team, and from what I've seen their gettext works with most languages.

    And for interfacing it with Perl, Inline::C!

    -- 
            dakkar - Mobilis in mobile
    
Re: Writing multilingual perl applications?
by Kanji (Parson) on Mar 19, 2003 at 21:17 UTC

       Thanks that appears to be a perfect solution.

       It addresses all the areas that I've thought were going to be problematic, and appears to have had a lot of real world use too!

      Steve
      ---
      steve.org.uk
      The SQL part does not use SQL placeholders. I believe this kind of coding practice should not be promoted.
(dkubb) Re: (1) Writing multilingual perl applications?
by dkubb (Deacon) on Mar 19, 2003 at 23:21 UTC
Re: Writing multilingual perl applications?
by adrianh (Chancellor) on Mar 19, 2003 at 20:58 UTC
Re: Writing multilingual perl applications?
by iguanodon (Priest) on Mar 20, 2003 at 03:10 UTC
    Where I work, we do this by storing the localized messages in a database. We have a module that provides a method to fetch a message for the current user's preferred language, substituting place holders for non-translatable parts of the message. So all the print statements or template-fillers in our applications look like

    get_localized_messsage($app_id, $lang_id, 35, 'Joe Customer', 3);

    which would return 'Welcome back, Joe Customer. It's been 3 days since you visited our site.' provided that message number 35 (in English) for this application is 'Welcome back, {0}. It's been {1} days since you visited our site.'. The place holders can be moved around by the translators to accommodate the syntax of different languages.

Re: Writing multilingual perl applications?
by Heidegger (Hermit) on Mar 20, 2003 at 08:17 UTC

    I make my Perl apps multilingual too. I have a Dictionary.pm module, which I wrote myself that works something like:

    my $dict = new Dictionary(); # Dictionary is made a singleton. $some_string = $dict->trans("word");

    My dictionary also has methods like $dict->check_in("word") that changes the meaning of a word if it exists or adds a new one.

    Once you have a module you can implement the methods the way you want and you can change them later if you don't like it. I had two options for Dictionary internals when I started working on it:

    First. Have XML dictionary files for each language: en_dictionary.xml, lt_dictionary.xml etc. Load the needed in the Dictionary constructor or change it at run-time. The problem with this option was that my app uses a windows-1257 charset and not utf-8. And the XML::Simple and other XML modules work only with utf-8 so far. So I dropped this option.

    Second. Have a database table dictionary with fields: word, en, lt for it. The dictionary justs loads the needed field values when needed.

    So, my way of doing might have been a reinvention of the wheel. However, my quick research on dictionary modules out there didn't give me good results. Conclusion: write a simple Dictionary wrapper around and choose a way to access your dictionary words inside it. You will be able to change it later when need. In your code you'll have delegation to the dictionary module: $dict->trans("word").

    I thought about finishing my module properly and putting it on CPAN. Don't know yet if it's good enough and really needed for Perl community.

Re: Writing multilingual perl applications?
by Corion (Patriarch) on Mar 20, 2003 at 08:26 UTC

    One thing that I stumbled onto recently was, how the Gimp handles internationalization - through an interesting hack :

    They (or rather, Marc Lehmann) introduce a special sub _ (underscore), and that sub does the lookup on whole phrases. This keeps the source code relatively clean, as your code is not cluttered with translate_message() calls, but every (translatable) string is prefixed by _ "foo".

    There are some complications to this matter, as the automagic source code scanning only works for _ "..." and not for qq{} constructs, also, it dosen't allow for any context-sensitive stuff like 1 file found vs. 2 files found. It allows for sprintf placeholders, something I urge you to allow too, as the construction of sentences varies enough to make hardcoded string construction ugly/awkward for the user.

    This sentence based approach has the drawback that you have to scan the source code and then translate all those sentences, but it has the advantage that you keep readable (english) error messages in your code, and these can be used as the defaults, if no translation is found.

    perl -MHTTP::Daemon -MHTTP::Response -MLWP::Simple -e ' ; # The $d = new HTTP::Daemon and fork and getprint $d->url and exit;#spider ($c = $d->accept())->get_request(); $c->send_response( new #in the HTTP::Response(200,$_,$_,qq(Just another Perl hacker\n))); ' # web

      This sounds like what UseMod does. Anyhow, its free software so I'll quote from it for a moment.

      # This could have alternate languages if extended. %Translate = ( 'Edit text of this page' => 'Edit this page', 'Index' => 'All Pages' ); # ... T('showing current revision instead') .... sub T { my ($text) = @_; if (1) { # Later make translation optional? if (defined($Translate{$text}) && ($Translate{$text} ne '')) { return $Translate{$text}; } } return $text; } # &ReportError(Ts('Could not create %s', $DataDir) . ": $!"); sub Ts { my ($text, $string) = @_; $text = T($text); $text =~ s/\%s/$string/; return $text; } # Improved Ts sub Ts { my ($text, $string) = @_; return sprintf $text, $string; }
Re: Writing multilingual perl applications?
by crouchingpenguin (Priest) on Mar 20, 2003 at 14:44 UTC

    Maybe not the best solution, but you can create a language HoH that contains the translations for different languages. For example:

    my $languages = { english => { not_found => 'File not found', }, german => { not_found => 'File nichte founden', }, }

    Then in your application you can set the default language like so:

    my $language = $languages->{german}; print $language->{not_found};

    cp
    ----
    "Never be afraid to try something new. Remember, amateurs built the ark. Professionals built the Titanic."
Re: Writing multilingual perl applications?
by DarknessX (Scribe) on Mar 19, 2003 at 21:19 UTC
    Just for random information, 'Nichte' translates to 'niece', which isn't what you want.. "Datei war nicht gefunden" works.

       Thanks - that just shows the futility of my token attempts to include another language!

       Thankfully other people will be making the translations once I've created the framework to make it possible.

      Steve
      ---
      steve.org.uk
Re: Writing multilingual perl applications?
by rinceWind (Monsignor) on Mar 20, 2003 at 11:24 UTC
    It's nice to see this topic being revisited. I posted this a year ago.

    Perl 5.8.0 has been released in the mean time, which has done much to resolve internationalisation issues.

Re: Writing multilingual perl applications?
by jonasbn (Scribe) on Mar 22, 2003 at 10:13 UTC
    I once had a similar problem which I solved with the following structure:
    %words = ( 'file_not_found' => { 'de' => "File nichte founden", 'dk' => "Fil ikke fundet", 'uk' => "File not found", } ); print $words{'file_not_found'}->{$lang}."\n";
    It makes it easy both add new languages and all you have to add is a 'translate' function which can look up words in the word list, based on phrases and a language indicator. I have the original code if you want to check it out.

       I don't see how this deals with the positional problem I referred to in my original post.

       but if you do have code to share perhaps posting it in the snippets section would be useful to others too?

      Steve
      ---
      steve.org.uk

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://244452]
Approved by adrianh
Front-paged by Kanji
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others examining the Monastery: (2)
As of 2024-04-20 03:12 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found