http://qs321.pair.com?node_id=11105105

Lotus1 has asked for the wisdom of the Perl Monks concerning the following question:

I have been given a set of batch files that use an ini file with environment variables defined like shown below. Every batch file calls a common bat file to read this ini and set local environment variables. I'm rewriting the batch into Perl but I'm trying not to change the configuration too much since folks are used to using it.

I wrote the following script to do environment variable expansion from the variables defined in the ini file. Is there a better way or a Perl Module that will do this? I'm hoping to not end up supporting this forever so I'm trying to use modules that will handle various edge cases I might not think of. This will need to handle things like spaces before and after the variable names which I haven't implemented yet. And I'll have to write something to handle garbled ini files.


use warnings;
use strict;

foreach(<DATA>){
    chomp;
    #       Skip blank or comment lines.
    next if /^\s*$|^\s*;/;
    #       Trim comments from the end of lines.
    s/\s*;.*$//;
    my ($key,$val)=split /\s*=\s*/;
    $ENV{$key} = $val;
    $ENV{$key} =~ s/%([^%]+)%/$ENV{$1}/g;
    print "$key = $ENV{$key}\n";
}

__DATA__
 
  ;
;comments
PROJECT_HOME=D:\Scripts\Projectname\ ;comments
package1=pack_test
package1_home=%PROJECT_HOME%%package1%

[download]

The output looks like this:

PROJECT_HOME = D:\Scripts\Projectname\
package1 = pack_test
package1_home = D:\Scripts\Projectname\pack_test

[download]

Replies are listed 'Best First'.
Re: Expanding environment variables from ini file
by swl (Parson) on Aug 27, 2019 at 06:27 UTC
[reply]

      Thanks, I should have mentioned in the OP that I looked at the INI modules I found but none mention environment variable expansion.

[reply]
Re: Expanding environment variables from ini file
by pryrt (Abbot) on Aug 27, 2019 at 13:41 UTC

    Using Config::Tiny, and assuming you don't actually need it to be in %ENV, but you just want to treat them as internal "environment-style" variables, and also want to accept true environment variables

    #!/usr/bin/perl -l
# in reponse to https://perlmonks.org/?node_id=11105105
use warnings;
use strict;

use Config::Tiny;
my $ini_string = do { local $/; <DATA> };
$ini_string =~ s/(?<=.);.*$//gmx;       # Config::Tiny cannot handle p
+ostfix comments

my $cfg = Config::Tiny->read_string($ini_string);

#use Data::Dumper; $Data::Dumper::Indent = 1;
#print Dumper $cfg;

# the _ key holds the main-category / top-level of your .ini file;
#   if you have multiple categories, this loop will have to be modifie
+d
foreach my $key (keys %{$cfg->{_}}) {
    while( $cfg->{_}{$key} =~ /\%(.*?)\%/g ) {
        my $var = $1;
        next unless exists $cfg->{_}{$var} or exists $ENV{$var};
        #print "\$var = >>$var<<\n";
        my $repl = exists $cfg->{_}{$var} ? $cfg->{_}{$var} : $ENV{$va
+r};
        $cfg->{_}{$key} =~ s/\%$var\%/$repl/;
    }
}

#print Dumper $cfg;
print $cfg->write_string();

__DATA__
;comments
PROJECT_HOME=D:\Scripts\Projectname\ ;comments
package1=pack_test
package1_home=%PROJECT_HOME%%package1%
subtemp=%TEMP%\subdir

    [download]

    On mine, that output 

    PROJECT_HOME=D:\Scripts\Projectname\
package1=pack_test
package1_home=D:\Scripts\Projectname\pack_test
subtemp=C:\Users\pryrt\AppData\Local\Temp\subdir

    [download]

    update: finished my first sentence; sorry

[reply]
[d/l]
[select]
      >>Using Config::Tiny, and assuming you don't actually need it to be in %ENV, but you just want to treat them as internal "environment-style" variables, and also want to accept true environment variables

      Your assumptions are mostly correct. Eventually I don't really need to use %ENV, but while I'm still using a mix of batch and Perl scripts I plan to keep using it.

      I like this approach of using Config::Tiny to parse the ini and then do the substitutions in my code. I've had good luck with Config::Tiny before so I'm already familiar with it. I can let it handle validation and parsing.

[reply]
[d/l]
Re: Expanding environment variables from ini file
by LanX (Saint) on Aug 27, 2019 at 10:23 UTC
    > will handle various edge cases I might not think of.

    There is an extremely stable approach I used on linux to "import" the ENV from a child process.

    see code samples in Re^3: How to "source" a shell file in Perl? (Trojan Dump)

    You might be able to adapt it for win/batch.

    • Your top-level Perl script basically calls a child-batch on second level which creates the environment variables.°
    • The batch itself calls another Perl one-liner on third-level which dumps/serializes* %ENV to STDOUT.
    • The top-level script reads from STDIN and evals the data

    No parsing whatsoever needed, just nested calls using standard mechanisms and modules.

    HTH! :)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery FootballPerl is like chess, only without the dice

    update

    *) using Data::Dumper ,which is in core

    °) by reading your INI or whatever your process is requiring

[reply]

      Why do you need that third level?

      On unixish OSes, the env command outputs the contents of the environment (as strings, so beware of newlines) and on Windows, the set command does that. Or did you output %ENV as JSON or something?

[reply]
[d/l]
[select]
        ...beware of newlines

        This should be a sufficient warning to avoid such a solution. There's no chance to find out whether the following env output comes from two environment variables FOO and PERL5LIB or just from setting one variable with export FOO=$(echo -e "bar\nPERL5LIB=my/malicious/stuff"):

        FOO=bar
PERL5LIB=/my/malicious/stuff

        [download]
        On Windows, the same misleading output can be created by typing the following (where <enter> is the "enter" key): 
        SET FOO=BAR^<enter>
<enter>
PERL5LIB=c:/my/malicious/stuff<enter>

        [download]
[reply]
[d/l]
[select]
        Because Data::Dumper creates a clean serialization of Perl data structures which can be eval'ed directly without any parsing on the top-level.

        update

        > Or did you output %ENV as JSON or something?

        kind of "something", a Perl dump from Data::Dumper which is core (contrary to JSON)

        Cheers Rolf
        (addicted to the Perl Programming Language :)
        Wikisyntax for the Monastery FootballPerl is like chess, only without the dice

[reply]

      That is a clever and interesting approach to that problem. For my situation I am able to use a function in a module I created to set the variables in the environment at the top level script. My predecessor had the batch files read the ini over and over but I'm planning to just test to see if a key is already set before re-reading the ini file.

      package custom_environment;
use strict;
use warnings;
use Exporter;

our @ISA = qw(Exporter);

our @EXPORT = qw( setup_modelbuild_environment );

####################################
### Setup environment variables from the mm config file.
###
sub setup_modelbuild_environment {
    $ENV{test_setup_vars} = "test123";
    print "sub setup_modelbuild_environment : test_setup_vars = $ENV{t
+est_setup_vars}\n";
}

1;

      [download]
[reply]
[d/l]

Back to Seekers of Perl Wisdom