Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?


by Mr. Muskrat (Canon)
on May 01, 2002 at 16:22 UTC ( #163292=sourcecode: print w/replies, xml ) Need Help??
Category: CGI Programming
Author/Contact Info Matthew Musgrove

Okay, this was created as a custom script. It is designed to read lines from a data file and "auto-submit" the data in the line to a cold fusion app. It works as advertised when run as a CGI script.

When run as from the command line, the data cannot be sent to the cold fusion application. You will see the line read, and the extracted fields. It will cycle through until there is no more data.

You may now use, modify, distribute, trash and generally do whatever you want with this software. My apologies to the company that paid for the last version of this. (Wow! Was it really almost two years ago?) You will not be reimbursed as it worked as promised and saved untold thousands of man-hours.

Versions prior to 3.0 used here documents to produce the HTML.

Updated: new version! 2002-05-08 12:30 (GMT -0600)

Updated: fixed email address 2002-05-07 13:21 (GMT -0600)


# Autosubmit version 3.3
# Last updated: 2002-05-07 22:25 (GMT -0600)
# Copyright, 2000 - present, Matthew Musgrove,

use strict;
use warnings;
use File::Copy;
use CGI;
use Text::ParseWords;

my $online;
$online = 1 if (defined $ENV{"REMOTE_ADDR"});            # assumes tha
+t the REMOTE_ADDR environment variable isn't set unless run as a CGI
our $q = new CGI;                        # so I'm lazy...
my $datapath = ".";                        # data path
my $filename = "data.txt";                                   # filenam
+e of data file
my $datafile = "$datapath/$filename";                # full file path 
+of data file
my $backup = "$datafile.bak";                    # full file path of b
+ackup file
my $header = "header.html";                    # header file
my $footer = "footer.html";                    # footer file
#my $action = "http://servername/path/to/cfm/enter_action.cfm";    # w
+here to submit the form, in this case a cold fusion app
my $action = "test.cgi";    # where to submit the form, in this case a
+ test script that spits out what you submit
my $FLock = 0;                                          # use file loc
my $timeout = 10000;                        # time (sec*1000) before s
+tarting next submission
my $delimit = '\|';                        # datafile delimiter, in th
+is case the pipe symbol

copy($datafile, $backup) if (! -e $backup);
my @data = ReadFile($datafile, $FLock);                # Read the data
+ from the file and return it sorted
my $size = @data;                        # Size of the array?
my $line;                            # One line from the array
if ($size > 0) {                        # At least one line in array
    @data = RemoveBlanks(@data);                # Return the non-blank
+ lines
    ($line, @data) = GetLine(@data);            # Get the last line an
+d the shorter array
    UpdateFile($datafile, $FLock, @data);                # Update the 
+datafile with the new data array
    CreateForm($header, $footer, $line, $delimit, $timeout, $action); 
+# create the frameset and turn it loose!
    exec("perl $0");                    # execute myself! using perl f
+or those systems that don't know what to do with the file
} else {                            # Cleanup
    unlink $header;                        # delete the header file, n
+o error checking because it's not critical
    unlink $footer;                        # delete the footer file, d
    if ($online) {
        print $q->header,
            $q->start_html('Job Finished'),
                "Back to the ",
                $q->a({-href=>'index.html'}, 'index'),
    } else {
        print "Job Finished.\n";
        unlink $datafile;                    # delete the original (no
+w empty)
        move($backup, $datafile);                # Restore the origina

sub ReadFile{
    my ($datafile, $FLock) = @_;
    open(DATA, "<$datafile") or die "Couldn't open $datafile for readi
+ng: $!\n";
    flock (DATA, 2) if ($FLock);
    my @data = <DATA>;
    return sort @data;

sub RemoveBlanks{
    my @data;
    foreach (@_){
        push(@data, $_) unless /^\s+$/;
    return @data;

sub GetLine{
    my (@data) = @_;
    my $line = pop(@data);
    return $line, @data;

sub UpdateFile{
    my ($datafile, $FLock, @data) = @_;
    open(DATA, ">$datafile") || die "Unable to update the datafile, st
+opped $!";
    flock(DATA, 2) if ($FLock);
    print DATA @data;

sub CreateForm{
# Create the header and footer files, then the frameset to contain the
    my ($header, $footer, $line, $delimit, $timeout, $action) = @_;
    CreateHeader($header, $line, $timeout);
    CreateFooter($footer, $line, $delimit, $action);
    print     $q->header(-expires=>"now"),
        "<html><head><meta name='pragma' content='no-cache' /><title>A
            $q->frame({-name=>"header", -src=>"$header"}),
            $q->frame({-name=>"footer", -src=>"$footer"})
        "</html>" if ($online);

sub CreateHeader{
# Create the header file.  It uses javascript to restart this web app.
    my ($header, $line, $timeout) = @_;
    if ($online) {
        open(HTML, ">$header");
        print HTML $q->start_html(
                                 -code=>'function DoIt(){    document.
+auto.submit();        }'},
                            -head=>$q->meta({-name => 'pragma',
                                        -content    => 'no-cache'}),
                             -onLoad=>"setTimeout('DoIt()', $timeout)"
+, ),
            $q->p("Line to process: $line"),
            $q->p($q->big('Wait for the page that says "Job Finished."
    } else {
        print "Line to process: $line\n";

sub CreateFooter {
# Create the footer.  It uses javascript to autosubmit the form to the
+ cold fusion app.
    my ($footer, $line, $delimit, $action) = @_;
    my ($ID, $title, $cat, $desc) = parse_line($delimit, 0, $line);  #
+ parses the line and puts the section into variables
    if ($online) {
        open(HTML, ">$footer");
        print HTML $q->start_html(
                                 -code=>'function DoIt(){    document.
+EnterForm.submit();        }'},
                        -head=>$q->meta({-name => 'pragma',
                    -content    => 'no-cache'}),
                            -onLoad=>"DoIt()", ),
                    $q->td($q->b('File ID')),
                    $q->td($q->textfield(-name=>"file_id", -value=>$ID
+, -maxlength=>11)),
                    $q->td($q->textfield(-name=>"title", -value=>$titl
+e, -size=>50)),
                    $q->td($q->textfield(-name=>"category1", -value=>$
+cat, -size=>50)),
                    $q->td($q->textarea(-name=>"description", -cols=>"
+20", -rows=>"3", -default=>$desc)),
                    $q->td($q->b('Convert to')),
                    $q->td($q->radio_group(-name=>'Convert', -values=>
+['HTML','PDF','No'], -default=>'No', -linebreak=>'true')),
                    $q->td($q->checkbox(-name=>'chkPrivate', -label=>'
+Check here if this is a private item')),
                    $q->td($q->b('Site ID')),
                    $q->td($q->checkbox_group(-name=>'site_id', -value
+=>['Department', 'Practice Group', 'Office'], -linebreak=>'true')),
    } else {
        print "ID = $ID\nTitle = $title\nCategory = $cat\nDescription 
+= $desc\n";
Replies are listed 'Best First'.
Re: Autosubmit
by TStanley (Canon) on May 01, 2002 at 18:22 UTC
    As a way of handling different delimiters, I would suggest that you use the Text::xSV module, which happens to be authored by the missing monk

    "Suppose you were an idiot... And suppose you were a
    member of Congress... But I repeat myself." -- Mark Twain

      After having carefully read through the docs for Text::xSV I have determined that while it is useful in most cases it is useless in *this* case. You have to have either a fields declaration line or fixed widths. Fixed widths is out for my program... just look at the data. In order to use a field declaration line, I would have to rewrite a major protion of the program.

      I have instead used Text::ParseWords which is not great but better than a simple split.

      I have also changed it so that the data set can be tested by running it from the command line. ;)

      The updated code is here.

      Matthew Musgrove
      Who says that programmers can't work in the Marketing Department?
      Or is that who says that Marketing people can't program?
Re: Autosubmit
by Mr. Muskrat (Canon) on May 01, 2002 at 18:35 UTC

    When I originally wrote it, the delimiter was a pipe but I thought that I'd put it into a variable just in case it the code was ever used by someone else. But now that I'm setting it loose for all to play with, I will have to look that up and see about incorporating it in.

    Matthew Musgrove
    Who says that programmers can't work in the Marketing Department?
    Or is that who says that Marketing people can't program?
      I've printed out the pod from Text::xSV and will be reading through it tonight. Perhaps I'll have time to code it in. Heck, I might just get around to the rest of my "want to do" list as well.

      Update: See this node for the follow up.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: sourcecode [id://163292]
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others taking refuge in the Monastery: (5)
As of 2021-01-24 21:38 GMT
Find Nodes?
    Voting Booth?