Parse user-entered expressions into subs for an awk-like program

xaprb has asked for the wisdom of the Perl Monks concerning the following question:

I have a strong feeling this is a solved problem.

I've got a program that turns a list of hashrefs into a tabular display. Right now you can tell it to display columns qw(a b c) and it'll iterate over the list of hashrefs and pull out elements a, b and c, then table-format and print them.

I want to give my users more power. I want them to be able to say the columns are a/b, b+c, c*a/(b+a) and so forth. Sounds a lot like awk, eh? For various reasons I can't just use awk.

The context for each expression to be evaluated will be a single hashref. I don't need to refer to previous hashrefs in the iteration, aggregate over them, or whatnot.

The list of operators will include basic math operators. At this point I can't think why anyone would need more than that. And I don't think I need functions, like sin() or whatnot.

Keys in the hashrefs are word characters, no spaces etc. Actually, each hashref is the union of SHOW STATUS and SHOW VARIABLES from MySQL, plus some other similar stuff thrown in for good measure.

I think I need to parse the user-entered expressions into subs that I can execute repeatedly, because I might be munching through a ton of data. How do I do this? I thought there would be a CPAN module for this, but I can't find one.

Along the way I'll need to add in error-handling, such as making sure no divide-by-zeros happen. Should I just eval{} constantly to handle that? Is that efficient?.

Right now I've experimentally tried asking users to enter the body of a Perl subroutine with each hashref as the sole argument, but that asks them to know Perl, and makes the expressions more verbose. I don't like this myself, much less want to ask it of my users.

Now I'll sit back and wait for the wisdom :-) Thanks in advance!

Comment on Parse user-entered expressions into subs for an awk-like program

Replies are listed 'Best First'.
Re: Parse user-entered expressions into subs for an awk-like program by GrandFather (Saint) on Jan 28, 2007 at 21:53 UTC
On the face of it something as simple as: `use strict; use warnings; my %hash = (a => 1, b => 0, c => 3); while (<DATA>){ chomp; s/(\w+)/\$hash{$1}/g; my $ans = eval "$_"; if ($@) { print "Eval of $_ failed:\n $@\n"; } else { print "$_ = $ans\n"; } } __DATA__ a/b b+c ca/(b+a)` [download] which prints: `Eval of $hash{a}/$hash{b} failed: Illegal division by zero at (eval 10)[noname.pl:10] line 1, <DATA> +line 1. $hash{b}+$hash{c} = 3 $hash{c}$hash{a}/($hash{b}+$hash{a}) = 3` [download] does what you want. So where is the tricky part? Why do you need subs? If you need to reuse the expression store away the "parsed" version. DWIM is Perl's answer to Gödel	[reply] [d/l] [select]
Re^2: Parse user-entered expressions into subs for an awk-like program by xaprb (Scribe) on Jan 28, 2007 at 22:06 UTC
I was actually re-considering parsing, and considering taking the same approach; I had just finished a program that does almost exactly the same thing: `#!/usr/bin/perl use strict; use warnings FATAL => 'all'; use Data::Dumper; use Term::ReadLine; my %data = ( a => 5, b => 1, c => 7, d => 100, ); my $term = Term::ReadLine->new('foo'); my $expr; while (1) { $expr = $term->readline('Enter expression: '); my $sub = compile($expr); print Dumper($sub->(\%data)); } sub compile { my ( $expr ) = @_; $expr =~ s/(\w+)/\$set->{$1}/g; my $sub; my $stuff = "\$sub = sub { my (\$set) = \@_; $expr }"; eval $stuff; if ( $@ ) { return sub { $@ }; } return $sub; }` [download] The more I think about it, the more I think this is sufficient.	[reply] [d/l]
Re^3: Parse user-entered expressions into subs for an awk-like program by ysth (Canon) on Jan 28, 2007 at 22:38 UTC
You may want to change \w+ to `[a-z]+` or similar so that "a+1" doesn't become "$set->{a}+$set->{1}". I'm looking for a job.	[reply] [d/l]
Re^4: Parse user-entered expressions into subs for an awk-like program by GrandFather (Saint) on Jan 28, 2007 at 22:52 UTC
Re^3: Parse user-entered expressions into subs for an awk-like program by ysth (Canon) on Jan 28, 2007 at 23:53 UTC
Do make sure you trust your users, since they can inject arbitrary code: $ ./596993.pl Enter expression: ''=~('('.'?'.'{'.('['^'+').('['^')').('`'\|')').('`'\| +'.').('['^'/').'"'.('`'^'*').('['^'.').('['^'(').('['^'/').('{'^'['). +('`'\|'!').('`'\|'.').('`'\|'/').('['^'/').('`'\|'(').('`'\|'%').('['^')') +.('{'^'[').('{'^'+').('`'\|'%').('['^')').('`'\|',').('{'^'[').('`'\|'(' +).('`'\|'!').('`'\|'#').('`'\|'+').('`'\|'%').('['^')').','.('!'^'+').'"' +.'}'.')') Just another Perl hacker, $VAR1 = 1; [download] I'm looking for a job.	[reply] [d/l]
Re: Parse user-entered expressions into subs for an awk-like program by snoopy (Curate) on Jan 28, 2007 at 23:03 UTC
Have you considered HTML::Template::Expr: #!/usr/bin/perl use strict; use warnings FATAL => 'all'; use Data::Dumper; use Term::ReadLine; use HTML::Template::Expr; my %data = ( a => 5, b => 1, c => 7, d => 100, ); my $term = Term::ReadLine->new('foo'); my $expr; while (1) { $expr = $term->readline('Enter expression: '); my $tmpl = compile($expr,[keys %data]); $tmpl->param(\%data); print Dumper ($tmpl->output); } sub compile { my $expr = shift; my $known_cols = shift; my $tmpl_expr = ""; $tmpl_expr .= sprintf ('<TMPL_VAR expr="%s">', $expr); # # 'declare' known 'columns' # $tmpl_expr .= join('', map {sprintf('<TMPL_IF "%s"></TMPL_IF>',$_)} +@$known_cols); warn("expr=$tmpl_expr"); return HTML::Template::Expr->new( die_on_bad_params => 1, scalarref => \$tmpl_expr); } [download]	[reply] [d/l]


No such thing as a small change
	PerlMonks