Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Substituting tokens into configuration values

by dbush (Deacon)
on Mar 20, 2003 at 13:18 UTC ( #244581=perlquestion: print w/replies, xml ) Need Help??

dbush has asked for the wisdom of the Perl Monks concerning the following question:


I appear to be suffering from brain fade today and so I'm having a bit of difficulty starting what I feel should be a simple problem. I wonder if anyone with a working brain can help?

Background: I am trying to write a Win32 service using PerlSvc from ActiveState to achieve something like this SOAP in an NT Service but as I can't predict what I will be running with it later, I wish to have the service driven by a configuration file.

Problem: I wish to enhance my configuration file with "tokens" that refer to other sections of the ini file.

This service has only the following purposes in life:

  1. Read an ini file (Config::IniFile).
  2. Start up a number of processes defined in it (Win32::Process)
  3. Check that the process don't go away over time (Win32::Process)
  4. When the time comes (i.e. the service is cancelled) shut the processes down again (Win32::Process).

The ini looks like this:

[GENERAL] scheduledb=e:\users\dbush\schedule.db [SERVICE] debug=1 interval=5 start1=browser.exe;browser;NORMAL_PRIORITY_CLASS start2=schedule.exe;schedule "localhost:%general-scheduledb%";NORMAL_P +RIORITY_CLASS

The format of the start lines is appname;cmdline;priority.

I have the basics of this, reading the .ini, starting the processes, monitoring then and then shutting them, working well and am happy about this. The problem comes that I wish to extend the command line section to include "tokens" e.g. %general-schedule% that can be parsed to include other configuration items e.g. e:\users\dbush\schedule.db.

After briefly contemplating using one of the templating modules (node Best templating system? and various others) I'm wondering if this is a bit of a sledgehammer approach? Would a better approach be to extract the tokens, retrieve the values and then substitute them back in? I had a Super Search but with little success.

Any help appreciated.


Replies are listed 'Best First'.
Re: Substituting tokens into configuration values
by dakkar (Hermit) on Mar 20, 2003 at 13:36 UTC

    I'd do it like this:

    The INI

    [GENERAL] scheduledb=something [SERVICE] start2=stuff ${GENERAL/scheduledb=default} etcetera

    The "getter":

    sub x_val { my ($ini,$sect,$key,$def)=@_; # same signature as 'val' my $val=$ini->val($sect,$key,$def); my ($sect1,$key1,$def1,$t); while ($val=~/\$\{([^}]+)\}/) { $t=$1; ($sect1,$key1,$def1)=($t=~m|^([^/]+)/([^=]+)(?:=(.*))?$|); $val=$`.(x_val($ini,$sect1,$key1,$def1)).$'; } $val; }


    • It uses pre- and post-match variables, so it will slow you regexps
    • The syntax is a bit different from the one you show
    • It expands recursively
    • It supports in-line defaults
    • I haven't tested it...

            dakkar - Mobilis in mobile
Re: Substituting tokens into configuration values
by zby (Vicar) on Mar 20, 2003 at 13:34 UTC
    For me the simplest approach to configuration files is to use perl syntax - and than directly build the datastructures by an eval. The substitution problem would be solved by an asignemnt in the built datastructure - something like this:
    $config->{general-schedule} = $general-schedule-config
    Where the $general-schedule-config was built by an eval as well.

      You can also use do to accomplish the same thing. Here is a simple example:

      #!/usr/bin/perl use strict; use warnings; use Data::Dumper; our $config; do '/tmp/'; print Dumper(\$config),"\n";

      and /tmp/ looks like:

      $config = [ option1 => 'one', option2 => 'two', option3 => 'three' ];

      It might also be advantagous to setup a signal handler (or whatever means/mechanism available as a win service) that will reread the config.

      "Never be afraid to try something new. Remember, amateurs built the ark. Professionals built the Titanic."
Re: Substituting tokens into configuration values
by Jenda (Abbot) on Mar 20, 2003 at 14:06 UTC

    If you would not insist on Config::IniFile ...

    use Config::IniHash; $config = ReadINI( 'Service.ini', systemvars => 0, # do not expand system %variables% case => 'preserve', # lookup is case insensitive. keys %$INI return the original case # ! if you change this option to a case sensitive one even the %se +ction-name% variables will be case sensitive ! forValue => sub { my ($name, $value, $sectionname, $INI) = @_; $value =~ s/%(\w+)-(\w+)%/$INI->{$1}->{$2}/g; return $value; }, ); use Data::Dumper; print Dumper($config);

    As you see you get a HoH containing the data from the INI file and you can "preprocess" the values before inserting them to the HoH.

    HTH, Jenda

    P.S.: To continue the shameless plug ... did you consider using PerlApp and Win32::Daemon::Simple?

Re: Substituting tokens into configuration values
by hiseldl (Priest) on Mar 20, 2003 at 14:32 UTC

    If you are not stuck using Config::IniFile and can switch to Config::General, then you get variable substitution built in.

    use Config::General; use Data::Dumper; $conf = new Config::General( -ConfigFile => 'y.cfg', -InterPolateVars => 1, ); my %config = $conf->getall; print Dumper(\%config); __END__ # config file contents # GENERAL # scoping allows substituting in the same block or # a nested block, see N.B. below. scheduledb=e:\users\dbush\schedule.db <SERVICE> debug=1 interval=5 start1=browser.exe;browser;NORMAL_PRIORITY_CLASS start2=schedule.exe;schedule "localhost:$scheduledb";NORMAL_PRIORITY_C +LASS </SERVICE> ########### Output: $VAR1 = \%config ### note how $scheduledb is substituted into 'start2' $VAR1 = { 'SERVICE' => { 'start1' => 'browser.exe;browser;NORMAL_PRIOR +ITY_CLASS', 'start2' => 'schedule.exe;schedule "localhost +:e:\\users\\dbush\\schedule.db";NORMAL_PRIORITY_CLASS', 'interval' => '5', 'debug' => '1' }, 'scheduledb' => 'e:\\users\\dbush\\schedule.db' };

    N.B. One thing to watch out for is scoping your name-value pairs. For all the details about scoping for interpolation, check out Config::General::Interpolated.

    What time is it? It's Camel Time!

Re: Substituting tokens into configuration values
by robartes (Priest) on Mar 20, 2003 at 14:03 UTC
    Here's a naieve implementation based on a hash lookup and a regexp.
    #!/usr/local/bin/perl -w my %tokens= ( 'general-scheduledb' => 'e:\users\dbush\schedule.db', 'animal-type' => 'mammal', 'species' => 'camel', ); my $inistring='start2=schedule.exe;schedule "localhost:%general-schedu +ledb%";NORMAL_PRIORITY_CLASS;%species%'; $inistring =~ s/%([^%]+)%/$tokens{$1}/eg; print $inistring; __END__ start2=schedule.exe;schedule "localhost:e:\users\dbush\schedule.db";NO +RMAL_PRIORITY_CLASS;camel
    I fudged your example string a bit to more clearly illustrate the approach.

    Note that this is a naieve implementation - you'll have to add checking that all tokens exist (as this code does not catch typo's in the ini file) for example.

    Update: Renamed %values hash to %tokens. Naming a hash values is asking for a headache :).

    Also, I missed the part that your token values are in the ini file as well. That means that using my implementation, you would build the %tokens hash from the [general] (or whichever) part of your ini file. However, you probably should start looking at modules for this, as jenda suggests below.


Re: Substituting tokens into configuration values
by dbush (Deacon) on Mar 20, 2003 at 17:42 UTC

    Many thanks (and a ++ tomorrow) to zby, crouchingpenguin, dakkar, robartes, Jenda and hiseldl for their insightful replies.

    The solution I have come up with is to use the tie functionality within Config::IniFiles and a slightly modified version of robartes regular expression approach. The while loop handles nested tokens (unlikely as they are in my case).

    The following is a code snippet of the approach I intend to use. Comments welcome.

    #perl -w use strict; use warnings; use Config::IniFiles; my ($appname, $cmdline, $priority, $szProcess); my %config; tie %config, 'Config::IniFiles', (-file => 'service.ini', -nocase => 1 +); my @processesToStart = map { $config{'service'}{$_} } grep { /^start[0-9]+/ } keys %{$config{'service'}}; foreach $szProcess (@processesToStart) { #Break process down into it's parts ($appname, $cmdline, $priority) = split /;/, $szProcess; #Process cmdline while ($cmdline =~ s/%([^%]+)-([^%]+)%/$config{lc $1}{lc $2}/eg) {} #Log and run it already print $cmdline, "\n"; } __END__ Text of service.ini: [GENERAL] scheduledb=e:\users\dbush\schedule.db [SERVICE] debug=1 interval=5 start1=browser.exe;browser;NORMAL_PRIORITY_CLASS start2=schedule.exe;schedule "localhost:%general-scheduledb%";NORMAL_P +RIORITY_CLASS Output: browser schedule "localhost:e:\users\dbush\schedule.gdb"

    The other modules mentioned look interesting (esp. Win32::Daemon::Simple) but I'm afraid I'm up against a bit of a deadline. I also didn't mention that the configuration file must be a standard ini as I share it with some C programs.

    Thanks again,

    Update: Corrected output typo.

      I would suggest one little change to the regexp:

      while ($cmdline =~ s/%([^%-]+)-([^%]+)%/$config{lc $1}{lc $2}/eg)
      This way it should be quicker since it does not force Perl to backtrack after it slurps up the whole general-scheduledb.

      Of course it's not entirely equivalent. If your INI contained %foo-bar-baz%, then your regexp would match it like %(foo-bar)-(baz)% and mine as %(foo)-(bar-baz)%. I don't think it matters.


Log In?

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://244581]
Approved by Tanalis
Front-paged by Jenda
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others about the Monastery: (4)
As of 2022-06-26 17:07 GMT
Find Nodes?
    Voting Booth?
    My most frequent journeys are powered by:

    Results (86 votes). Check out past polls.