|
Bod has asked for the wisdom of the Perl Monks concerning the following question:
I was recently chatting about writing a web-based system, and the idea came up of making the interface multi-lingual. Just to be clear, this was idle chit-chat and this is not something that will be built (probably!)...but it got me thinking...
If I were to build a web-based system, how would I make it multi-lingual? In other words, all the interfaces and other wording on the site could be set to display in another language instead of the one it was originally written in.
Let's leave aside the problem of actually doing the translations as that is the same regardless of implementation. Let's also assume that we are going to use Template to build the system.
The first option is to have the template files contain just structure and the language passages.
<h1>[% heading %]</h1>
<p>[% introduction %]</p>
<form method="post">
<table>
<tr>
<td class="label">[% form_name %]</td>
<td><input type="text" name="name"></td>
</tr>
</table>
</form>
Then to pass all the text parts to the template file from the Perl script.
This seems to violate the concept of MVC and involves passing huge numbers of template variables. It also makes the template file very difficult to read as there is not much text, only the variables.
The second way I came up with was to do much the same but to hold the template variables in a language template file and PROCESS that. There would be a different language template file for each language:
[% PROCESS $language.tt %]
<h1>[% heading %</h1>
<p>[% introduction %]</p>
Again this makes the template file very difficult to read.
Lastly I considered having completely separate template files for each language. Arrange these in different directories by language and pass in language-specific variables where variables would be passed in for a mono-lingual system.
This last approach has lots of advantages. The template files are easy to read as they have all the static text, Only the necessary variables are passed into Template and it is easy to add another language. But there is a big disadvantage - although it is easy to implement, it would be very difficult to maintain. A change made to one template file would need to be made to the corresponding one in every other language. It would probably need a script to keep them all in sync.
Is there another way that I haven't thought of?
I'm sure I'm not the first person to have thought this through!
What is the "best" solution to this problem?
|
Re: Multilingual design
by kcott (Archbishop) on Dec 11, 2022 at 02:49 UTC
|
G'day Bod,
I had to do this for $work about 20 years ago; I expect my memory is somewhat shaky on details by now.
Here's what I remember as it applies to your general design.
I believe there are now a number of applications which will do much of the heavy lifting;
it may be better to use those instead of re-inventing wheels yourself.
I'll be interested in what others suggest.
Firstly, just use one set of templates.
Improve readability with meaningful id="...", class="...", etc. names; remember you can use multiple class names.
This seems to align with what you suggested as your first option.
I don't see any MVC-violation; please explain more about what you meant.
Store translations in separate directories.
If you're going to include language variations, store the main language (let's say fr for French) in one place,
and then variations (e.g. fr_ca for Canadian-French) separately.
The variations would be smaller and only need to overwrite parts of the main language.
To use a Perl-based pseudocode (which isn't necessarily suggesting implemention):
%lang = (%lang_fr); # French
%lang = (%lang_fr, %lang_fr_ca); # Canadian-French
Organise translations into useful categories.
For instance, button text ("Submit", "Cancel", etc.) would be widely used and perhaps suitable for a "common" category;
text specifically for the "Left-handed Sky Hooks" pages could probably go into their own category.
Bear in mind that scripts can be left-to-right (e.g. Cyrillic & Greek)
while others are right-to-left (e.g. Arabic & Hebrew).
Your overall layout design would need to take this into account.
In some cases you could just the change direction of the text:
+-------------+ +-------------+
| LTR Lang | | gnaL LTR |
| Description | | noitpircseD |
+-------------+ +-------------+
Others may be a bit more complicated if controls need to be moved as well:
+---------+-----------+
| Prompt: | [_______] |
+---------+-----------+
+-----------+---------+
| [_______] | :tpmorP |
+-----------+---------+
There are other considerations related to scripts (e.g. fonts & encoding).
I'd suggest having a "meta" (or similar) category for each language to hold this information.
Navigation of a multi-lingual site needs to be addressed.
Here's a couple of examples of the types of issues that could be encountered:
-
I follow links through several pages in Urdu; switch to Brazilian-Portugese; hit ←.
Should I go back to the page in Urdu or should a see Brazilian-Portugese translation.
-
I select a page in Swahili that's part of a set.
I hit the "Next" button a few times to move through the set.
On one page, there is no Swahili translation for the next page in the set. What happens?
Hide the "Next" button? Show the next page in a fallback language?
Print messages of remorse in various places?
Pretend there's been a server error and hide under a rock?
[the level of silliness creeping in
is telling me that I've spent too much time answering this question]
As you can see, there's more involved than simply translating text.
Edit:
A few cosmetic changes; no information added, deleted, or substantially changed.
| [reply]
[d/l]
[select] |
|
|
When we did our first serious multi-lingual deployment, we also had to contend with languages which read top to bottom first, and then either right-to-left or left-to-right.
Just another (pardon the pun) dimension of complication.
| [reply] |
|
|
| [reply] |
|
|
Cheers Ken for the great reply...as always you get the thought process moving in the right direction!
I don't see any MVC-violation; please explain more about what you meant
In the first solution I came up with, I was suggesting passing the language specific text from the script to the template.
my $vars;
if ($lang eq 'en') {
$vars = {
'heading' => 'Dies ist eine Einführung',
'text' => 'Hier ist ein Beispiel für einige Inhalte',
'seemore' => 'mehr sehen...',
};
} elsif ($lang eq 'au') {
$vars = {
'heading' => 'G\'Day, this is an intro',
'text' => 'Hey Sport, just a little example',
'seemore' => 'want more...',
};
} else {
# Default to English
$vars = {
'heading' => 'This is an introduction',
'text' => 'Here is an example of some content',
'seemore' => 'see more...',
};
}
$template->process('intro.tt', $vars);
I wouldn't implement it like that, but it shows my thought process...
I quickly moved on from there and went for having the template variables in their own template file and using a PROCESS directive to set the correct language. I only included the first option so you could get a glimpse of how my mind dealt with the stages of developing the idea.
I'll come back to the rest of your post in a while. | [reply]
[d/l]
[select] |
Re: Multilingual design
by cavac (Parson) on Dec 16, 2022 at 09:57 UTC
|
My own webserver uses TT. Text in the templates is defined in English, but with a translate callback, something like this:
<h1>[% tr.tr("Hello World") %]</h1>
<p>[% tr.tr("Your new username is") %][% tr.quote(username) %]</p>
Basically, i'm registering a plugin into TT that provides a few functions. That includes making variables websafe as well to reduce the possibility of code injection. I haven't touched that particular code in a long, long time, but basically, you start by creating a TT plugin module
package PageCamel::Web::TT::Translate;
...
use base qw(Template::Plugin);
use Template::Plugin;
use Template::Exception;
sub load($class, $context) {
my $self = bless {
}, $class;
return $self;
}
sub new($self, $context) {
return $self;
}
sub tr($self, $data) {
return $data if($data eq '');
my $lang = $self->getLang;
my $trans = tr_translate($lang, $data);
return $trans;
<}
...
And then you provide that plugin when instatiating the Template Toolkit:
use Template;
...
my $tt = Template->new({
PLUGINS => {
tr => 'PageCamel::Web::TT::Translate',
},
});
And since you provide the original, untranslated text in the Template, you can return that as default when no translation is found. Plus, automatically add it to the "still to be translated" list.
Original codebase of mine is open source.
| [reply]
[d/l]
[select] |
|
|
| [reply] |
Re: Multilingual design
by alexander_lunev (Pilgrim) on Dec 11, 2022 at 19:45 UTC
|
Good evening!
I've never implemented multilingual desing in Perl projects - at least as an original idea or intent of those Perl projects - but have used to JavaScript vue-i18n library (fast guide) that utilizes third mentioned way of making multilingual site. There you have a JSON file for each language with all language strings as a hash, and language switch can be done by HTTP headers or some other things (like session).
Using language in template is like:
{
somepage: {
title: 'Hello, {userName}!'
}
}
<title>{{ $t('somepage.title', { name: 'Alexander' }) }}</title>
||
VV
<title>Hello, Alexander!</title>
At first this was a language file for frontend (written, as you can see, in Vue, or more precisely - in Quasar), but then it comes handy in Perl Mojolicous backend - I just read it as JSON to a hash, and voila: I can use these language strings to make ODF documents in current language, for example. So I vote for third way as I used it and this way can be convinient and language source can be used by another program - it's a simple JSON that converts from and to Perl data structures. Data once entered by human will remain readable and maintainable, as it's just a text file in all known format - this is what I like most in this scheme. | [reply]
[d/l] |
Re: Multilingual design
by Polyglot (Chaplain) on Dec 11, 2022 at 17:36 UTC
|
I won't claim to have all the answers. I'm a simpleton who cannot grasp, and therefore does not use, OOP. That said, many modules are beyond my comprehension as well, and I cannot use them.
However, I have developed multiple websites that were multilingual. If only a few languages are anticipated, it is simple enough to load them into a JavaScript array and switch languages "live" without a callback to the server. However, when the language data begins to become more weighty, what I have done is to store them in a database and use AJAX to retrieve them as necessary.
Here is an example for the code I use.
#LANGUAGE ARRAY IS DEFINED FIRST VIA PERL
#SOMETHING LIKE . . .
my %ilanguages = (
language => [ 'English','ไทย','ລາວ','Español','việtnam','简体中文','繁体中文','한국어' ],
submit => [ 'Submit','กดส่ง','ກົດສົ່ງ','Someter','gửi đi','提交','提交','' ],
preferences => [ 'Preferences','การตั้งค่า','ການຕັ້ງຄ່າ','Preferencias','sở thích','喜好','喜好','' ],
reposition => [ 'Reposition','เปลี่ยนตำแหน่ง','ປ່ຽນຕໍາແຫນ່ງ','Reposicionar','tái định vị','复位','复位','' ],
);
#THEN THE ARRAY IS USED TO HASH THE REQUESTED LANGUAGE COLUMN
#SOMETHING LIKE THIS...
sub interfaceLanguage {
my $language = shift @_;
unless ($language) { $language='English' };
if ($language eq 'Lao') { $language = 'ລາວ' };
if ($language eq 'Thai') { $language = 'ไทย' };
if ($language eq 'Spanish') { $language = 'Español' };
if ($language eq 'Vietnamese') { $language = 'việtnam' };
if ($language eq 'Korean') { $language = '한국어' };
if ($language eq 'Simplified Chinese') { $language = '简体中文' };
if ($language eq 'Traditional Chinese') { $language = '繁体中文' };
my $index = -1;
if ($language ne '') {
foreach my $lng (@{$ilanguages{'language'}}) {
++$index;
last if ($lng eq $language);
}
} else {
$index = 0;
}
for my $eachkey (keys %ilanguages) {
# %ilang PRE-DECLARED AS GLOBAL
$ilang{$eachkey} = $ilanguages{$eachkey}$index;
};
return;
} #END SUB interfaceLanguage
Then the HTML code is formed using the language "handle" for each required expression.
#BUTTON VALUE
<input type="button" id="resetwindows" onclick="setScreen();setpos();s
+etScreen();setpos();" value="$ilang{reposition}" />
So instead of making the value of your button say something like "Submit", you set the value to "$ilang{submit}". This will get replaced by whichever language is selected.
In the case of a form element in which its value is its text or label on the screen, the value can be updated via JavaScript without necessitating a page reload. If its value is stored in a JS array, this can be done entirely client-side. Or if there is other text, it can be placed in a <div id="preferences"></div> and updated to the language of choice via the innerHTML property for that div element, e.g. document.getElementById("preferences").innerHTML="$ilang{preferences}";. But if the site is to have many language options, AJAX becomes more attractive, and instead of the Perl array I excerpted as an example, a database table for the language data becomes more expedient.
One major advantage to this system is that it also facilitates translation into other languages. It would be simple to create a utility for adding a language column to the database. Adding the new language to the website would then be as simple as adding its name, and possibly a language flag-icon to represent it, to a language menu. Whichever language was selected, that column in the table would be found to supply the language items throughout the page.
| [reply]
[d/l]
[select] |
|
|
Many thanks for that Polyglot - your knowledge of languages is undoubtedly greater than mine. Although, I would have anticipated that from your username 😜
Your solution is workable. But I'm not sure about using AJAX instead of reloading the whole page given that pretty much every bit of content will change. It also suffers from violating the MVC model as the text to display is being handled by the controller (i.e. the Perl script) and not the view (the Template in my case). I do appreciate that your example prints the HTML directly from Perl and not through a template.
| [reply] |
|
|
I use AJAX instead of reloading the entire page because I am running the language conversion on interactive pages with web forms. A page reload will mean the user must reset and/or reenter all of his or her already-entered information again. But if you are only wanting to have static pages, there's no advantage to AJAX; it would make sense to just reload the page.
| [reply] |
|
|
|
|
|
|
