http://qs321.pair.com?node_id=524970

diego_de_lima has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks!

I'm probably facing a simple problem that many of you have already solved long time ago: how to store and read many translations of a software.

Our application is a Web 2.0 software running under Linux/Apache/mod_perl/Postgres/BerkeleyDB for cache/etc. This application has about 1000 different expressions (from 10 to 200 character each). And we are plannig to prepare it to have multiple languare support (each user that logs in can choose it's prefered lang, just like phpPgAdmin).

So my question is: which is the best way to store all this data.

My fisrt choice was to have it on simple PM files, on hashref structures, wich is the fastest way to access the data, but having multiple languages simultaniously on mod_perl would increase memory consumption, having a hash of thousands of expressions in memory when each CGI may use only a few of them.

My second choice was to have it chached on a BerkeleyDB file (which I already use for other caches), tied to a hash. This would solve the Apache/mod_perl memory problem, but is this fast enougth?

So, before I begin working, what are you monks doing to solve this? Any of my ideas or a different one?

Thanks!

Diego de Lima

2006-02-09 Retitled by planetscape, as per Monastery guidelines
Original title: 'Multi-languare web app'

Replies are listed 'Best First'.
Re: Multi-language web app
by thedoe (Monk) on Jan 23, 2006 at 17:18 UTC

    I would have to say this is one of the best types of examples for MVC architecture programming. By having a view subdirectory for each language, you would be able to look for the correct template file(s) to call based on that user's preference.

    I notice, however, that you mention BerkeleyDB. Does this mean that a lot of your site is dynamic content? If so, where does this information come from? Are you trying to translate that content as well, or only the static portions of your page?

    By having the majority of the output of your program defined within template files, you could solve most of your problems by only duplicating these. This will save on the mod_perl memory load. However, if your packages are the ones actually outputting the html, perhaps it is time to look at how that is done. After all, this sounds like a large change to your application. What better time to re-examine the structure of your application and see if it truly is still the best for what you are trying to do?

    Either way, simply adding PM files for each language does not seem to be the best solution to this problem. As you have already pointed out, the scalability factor just isn't there. Good luck, and I'd be happy to explain any of this in more detail if you like.

    Update: I am looking for links to help you out. There was an excellent discussion about what MVC really is about and how to implement it. I would definitely recommend reading through that.

Re: Multi-language web app
by ruoso (Curate) on Jan 23, 2006 at 17:18 UTC
Re: Multi-language web app
by Tanktalus (Canon) on Jan 23, 2006 at 17:31 UTC

    Where do you store your text now? If, as I would hope, you have your text in templates (whether using Text Template, HTML::Template, or another similar solution), those templates are going to be stored either on disk or in a database. If it's in a database, simply add a new column for language (I suggest using ISO-standard language identifiers, either 2-letter (en), or 4-letter-with-middle-underscore (en_US), or 4-letter-with-middle-underscore-dot-encoding-name (en_US.iso88591)). If it's on disk, then use the language identifier as a directory name instead.

    Also, don't forget that if a particular template is not found in the location you want, you need to be able to fall back to a reasonable default. e.g., if the desired files aren't found, and your default language is English, then fall back to English. If it's not there, then you have real problems ;-)

    I've always noticed that it's easier to design this in from the ground up than it is to add in after the fact.

      Well, thanks for the advices, it's getting clear to me.

      In fact, despite this is a web app, it's much more a software than a web page. So, the templates are quite similiar to each other (think of phpPgAdmin, it's very similar to that).

      More details:
      - I'm using a homemade MVC framework, very similar to Catalyst. But I want to have just ONE view/template to ALL supported languages in one page - the templates/views are quite complex, with lots os javascript, DHTML, Ajax...
      - The software is an ERP solution. So I don't care that much on beauty, but more on usability.
      - I have a 10 Gb PostgeSQL database behind it, 500 tables using one single template for browsing, other for details, etc... So, you see, this template must be quite complex to handle so many different field types.
      - I use BerkeleyDB with MLDBM to cache lots of user information, like passwords, and some postgres schema information, and it's doing a good/fast job on that (but I acsess it only a few times per request, not dozen of times like it would happen with a lang dictionary).
      - This time I'm not worryed about translating this 10 Gb DB :)
      - I'm worried about translating simple software messages like: "Please enter your username and password", "The required field %s is blank", etc...

      Yes I've seen linux gettext and I thought that's a good way too. Do you know if it's faster than BerkeleyDB?

      Thanks,

      Diego de Lima
Re: Multi-language web app
by pKai (Priest) on Jan 23, 2006 at 18:12 UTC
    1000 different expressions (from 10 to 200 character each)

    I have no idea, if memory consumption is really a problem in such a scenario.

    But managing the proper translations alone with all the possible pitfalls is a problem.

    For a framework to integrate multi language support you might have a look at Locale::Maketext.