As part of the analysis of a set of experiments that I do, there are several simply mathematical adjusts, DOS-based programs, and other tools that we can use on the raw data in order to develop usable results. In the past, we'd only collect < 20 of these data sets (simply x-y data, nothing fancy) at a time, and so hand-processing the data with these tools was rather simple.

However, we've gotten capabilities to do much more data processing, possibly looking as much as 300-400 sets from a run. Using the hand methods at this point doesn't seem like it's going to work well.

Fortunately, I have access to perl, so I've developed several small scripts that help me along. However, thinking about the big picture, I've been starting to 'wrap' the usage of the DOS programs into perl code, and also breaking apart the other parts of the perl code to smaller functions, such that I can write a very simply code to process my data. For example, my end goal would be to write something like this (in perl):

foreach my $dataset (1..50) {
    my $data = myread( "$filename.$dataset" );
    $data = shiftfunction( $data, $shiftvalue );
    $data = normalize( $data, { normconstant => 1 } );
    mywrite( $data, "$newfilename.$dataset" );
}
[download]

(Not actual function names, but the idea is there.)

This will be great for me, since perl's quite easy to use, but I'm thinking down the road that others will possibly want access to this data processing. And while I'm sure that others could learn and use perl (Remember, I'm in a chemical environment, not comp.sci.), I'm thinking that using a lightweight language would be easier for them to learn and pick up. For example, I'm considering the above script to be written in this microlanguage as:

for $i in 1-50 {
    $data = read "datafile.$i";
    shift $data 0.05;
    normalize $data constant=1.0;
    write $data "outfile.$i";
}
[download]

Note that every 'function' would be of the form <function_name> <required args> [<optional args in name=value pairs>]. There would be some basic variable knowledge, and there would be cases where one could access internals of variables using dot notation, instead of ->{} as with perl. The semi-colons would be used only because the name=value pair sets could be numerous for a given command. The only programming syntax that would be used would be the 'for' construct, at least at this stage.

Conceptionally, this doesn't seem hard. But there's obviously several gotchas that I haven't processed completely. But I'm wondering if anyone has done a similar task in developing a microlanguage, and could provide any pointers for writing one?

-----------------------------------------------------
Dr. Michael K. Neylon - mneylon-pm@masemware.com || "You've left the lens cap of your mind on again, Pinky" - The Brain
"I can see my house from here!"
It's not what you know, but knowing how to find it if you don't know that's important

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.