Beefy Boxes and Bandwidth Generously Provided by pair Networks
Your skill will accomplish
what the force of many cannot
 
PerlMonks  

Using perl to script an upload page

by Aldebaran (Curate)
on Jan 02, 2023 at 07:21 UTC ( [id://11149277] : perlquestion . print w/replies, xml ) Need Help??

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

Hi Monks

I've had some success with scripts lately, but unfortunately, I haven't been able to upload them to gitlab yet. I was hoping that I might change my luck by starting with a clean slate. I couldn't get my source scripts loaded up to gitlab repo for betelgeuse. I have expended more than 3 strikes thrice. Maybe a change of tactic....

I know I've run scripts from bliako before to create repos on github. I looked through his and my stuff and didn't see anything, nor do I get anything from a super search:

Found 0 nodes roughly between 2023-01-02 and 2012-01-20 (searched 26. +04% of DB). where any text contains all of "create", "repo"

So again, it's looking like a need to start from scratch. This project wants to be an upload page using perl. Let's start with some primitive html:

<!DOCTYPE html> <html> <head> <title>How to upload files using HTML to website?</title> </head> <body> <center> <h1>Welcome to GFG</h1> <h2>How to upload files using HTML to website?</h2> <input type="file" id="file1" name="upload"> </center> </body> </html>

I've spent some time now fishing around the net for best methods, and time before the advent of CE 2023 is very relevant. I wish I could set my browser to get me nothing that is over 5 years old. Heck I'm not too proud to ask:

OT Q1) How do you search the web with a secure anonymous browser that prioritizes recent matches and thinks anything 15 years old is broken?

I don't find a date on is position everything? and not completely sure about geeks for geeks. This part looks about right:

<form method=”post” enctype=”multipart/form-data”> <div> <label for=”profile_pic”> Choose file to upload </label> <input type=”file” id=”profile_pic” name=”profile_pic” accept=”.jpg, .jpeg, .png”> </div> <div> <button> Submit here </button> </div> </form>

Here's another take on the html:

<!DOCTYPE html> <html lang="en"> <head> <meta http-equiv = "Content-Type" content = "text/html; charset=ut +f-8" /> <title>File Upload</title> </head> <body style = "text-align:center;"> <h1 style = "color:green;"> GeeksForGeeks </h1> <form action = "/cgi-bin/upload.cgi" method = "post" enctype = "multipart/form-data"> <p>Upload Photo: <input type = "file" name = "photo" /></p> <p>Email: <input type = "text" placeholder = "e.g. GFG@gmail.com" name = "email_address" /></p> <p><input type = "submit" name = "Submit" value = "Submit Form" /> +</p> </form> </body> </html>

On the perl side, this is where I'm starting:

use CGI; $query = new CGI; $filename = $query->param('file-to-upload'); my ($bytes, $buffer); open (IMAGE, ">image.gif"); while ($byte = read($filename, $buffer, 1024)) { print FILE $buffer; Quote: } close (IMAGE); print "Content-type: text/html\n\n"; print "Done Uploading Image"; exit;

Q2) Do we live in a post CGI-bin world? This looks pretty ancient to be worried about buffers.

Q3) What are preferred and modern ways to do this? (If the answer is PHP, then I don't want to jump syntaxes to do this erstwhile trivial thing.)

This is where it always ends for gitlab now:

fritz@laptop:~/Documents/gitlab1$ ll total 52 drwxrwxr-x 4 fritz fritz 4096 Jan 1 21:24 ./ drwxr-xr-x 10 fritz fritz 4096 Jan 1 21:24 ../ -rwxr-xr-x 1 fritz fritz 1156 Dec 24 22:43 1.2.solstice.pl* ... drwxr-xr-x 2 fritz fritz 4096 Dec 15 21:44 conf_files/ drwxrwxr-x 8 fritz fritz 4096 Dec 25 11:07 .git/ -rw-rw-r-- 1 fritz fritz 14 Dec 19 13:58 README.md fritz@laptop:~/Documents/gitlab1$ git remote -v origin git@gitlab.com:tallharry66@yahoo.com/trans.git (fetch) origin git@gitlab.com:tallharry66@yahoo.com/trans.git (push) fritz@laptop:~/Documents/gitlab1$ git add *.pl fritz@laptop:~/Documents/gitlab1$ git push -u origin master ssh: Could not resolve hostname gitlab.com: Name or service not known fatal: Could not read from remote repository. Please make sure you have the correct access rights and the repository exists. fritz@laptop:~/Documents/gitlab1$

I do have this:

fritz@laptop:~/Documents/gitlab1$ ssh -T git@gitlab.com Welcome to GitLab, @tallharry66! fritz@laptop:~/Documents/gitlab1$

, but then I man ssh and realize, I don't even know what this means:

-T Disable pseudo-terminal allocation.

, so

Q4) How do experienced Gitlabers cook up a project from scratch?

Thanks for your comment,

Replies are listed 'Best First'.
Re: Using perl to script an upload page
by haukex (Archbishop) on Jan 02, 2023 at 08:18 UTC
    This looks pretty ancient to be worried about buffers.

    Well, CGI.pm is indeed pretty ancient (see also for modern alternatives), but its documentation does show that as an example of how to handle the upload. BTW, your code example appears to be missing strict and warnings, proper open usage, and a binmode (or '>:raw' layer in the open) on the filehandle.

    What are preferred and modern ways to do this?

    Personally, I prefer Mojolicious. Its tutorial includes an example, and on my scratchpad I have linked quite a few of my Mojo examples.

    ssh: Could not resolve hostname gitlab.com: Name or service not known

    If your ssh -T git@gitlab.com works on the same machine, then that's really strange and would indicate a problem with your network configuration or some other strange thing is going on. Though this is definitely worth debugging, I just wanted to take this oppertunity to say that I usually recommend the https protocol for all git remotes, as my experience has been it's easier to set up, especially in combination with e.g. git-credential-cache.

    How do experienced Gitlabers cook up a project from scratch?

    Though I personally take a different route, probably the easiest way is to create a new project on the website, leave the option "Initialize repository with a README" or similar enabled, and then do a git clone to get that repository onto the local machine, and then start working in this new directory, pushing changes up to the server as desired.

      Though I personally take a different route, probably the easiest way is to create a new project on the website, leave the option "Initialize repository with a README" or similar enabled, and then do a git clone to get that repository onto the local machine, and then start working in this new directory, pushing changes up to the server as desired.

      You know, haukex, I might read your posts ten times and still get something from them. I finally cottoned on to the whole git clone deal, and I'm able to move forward:

      $ git clone https://github.com/haukex/webperl.wiki.git wiki Cloning into 'wiki'... remote: Enumerating objects: 64, done. remote: Total 64 (delta 0), reused 0 (delta 0), pack-reused 64 Unpacking objects: 100% (64/64), done. $ ls 1.example.config.txt 1.txt 2.txt 4.getopt.pl 5.git.sh wiki 1.git.sh 2.git.sh 3.git.sh 5.getopt.pl README.md $ cd wiki $ ls Building-WebPerl.md _Footer.md Perl6.md Using-Web +Perl.md FAQ.md Home.md Post-your-Examples-Here.md $ cat Using-WebPerl.md These are **supplemental** notes on using WebPerl. Please see the main documentation at **<https://webperl.zero-g.net/usi +ng.html>**. - Nothing here yet. $ pwd /home/pi/Documents/curate/perlmonks/trans/trans/trans/wiki $ ls Building-WebPerl.md _Footer.md Perl6.md Using-Web +Perl.md FAQ.md Home.md Post-your-Examples-Here.md $ git clone --branch gh-pages https://github.com/haukex/webperl.git pa +ges Cloning into 'pages'... remote: Enumerating objects: 1106, done. remote: Total 1106 (delta 0), reused 0 (delta 0), pack-reused 1106 Receiving objects: 100% (1106/1106), 6.15 MiB | 2.48 MiB/s, done. Resolving deltas: 100% (711/711), done. $ ls Building-WebPerl.md _Footer.md pages Post-your-Examples-Here.md FAQ.md Home.md Perl6.md Using-WebPerl.md $ cd pages/ $ ls building.md favicon.ico legal.md regex.html CNAME HD_WebPerl_GPW_2019_Slides.pdf _misc using.md _config.yml HD_WebPerl_Riga_2019_Slides.pdf notes.md webperl.png democode index.md perl6.md $

      Q1) How do I clone the rpi repo that I visit so much and now can't find?

      I hadn't been in a position to follow webperl main page when you rolled it out. I remember that I had some crummy hosting arrangement at the time that didn't allow me to have sudo privileges, so I couldn't replicate much of any of it. Well, now is the dead of winter, and I have my perl feet under me now. I'm confident that if I typed:

      plackup webperl.psgi

      , your software would snap to attention and be ready for contact. So I have executed the cpanfile that was:

      # $ cpanm --installdeps . requires 'Cpanel::JSON::XS'; requires 'Plack'; requires 'Plack::Middleware::CrossOrigin'; requires 'Plack::Middleware::Auth::Digest';

      Q2) Is Plack the solution for my upload question?

      I'm concerned that I see the date of 2009 at stackoverflow thread on psgi and advent.plackperl, but I'm hoping that means that it is well-seasoned rather than obsolete. I am encouraged to see the name Miyagawa on it. One comment from these sources:

      I'm new in PSGI too, but zilion things now are much easier as before in my apache/mod_perl era. try write some apps with it and will find than Plack is like the first sex - now you didn't understand that you could live without it. ps: If was here something like "Perl Oscar", will sure nominating MyiagavaSan. :)

      That's kind of the way I feel about:

      cpanm --installdeps .

      Again, thanks for your writeup....

      Gruss aus Amiland,

        Q1) How do I clone the rpi repo that I visit so much and now can't find?

        My Raspberry Pi notes used to be in my "hgpstools" repository, and a few older notes are still there, but I've moved my main and most current Raspberry Pi notes to a separate repository, which is mirrored to GitHub: git clone https://github.com/haukex/raspinotes.git

        Q2) Is Plack the solution for my upload question?

        Plack is somewhat lower-level, but still relevant, and though I haven't used it myself for this purpose, it is one way to handle file uploads. I would still recommend Mojolicious instead - I don't understand your previous comments about it: "... deployment size and simplicity is relevant. I don't need a lot of software here." Mojo isn't all too big, and only has very few non-core dependencies, so it really isn't that big of a deal, and what you get in exchange is a really good framework.

      Well, CGI.pm is indeed pretty ancient (see also for modern alternatives), but its documentation does show that as an example of how to handle the upload. BTW, your code example appears to be missing strict and warnings, proper open usage, and a binmode (or '>:raw' layer in the open) on the filehandle.

      Thx, haukex, I finally got some results after digesting forum comments properly. I didn't mean for the CGI code for more than a motivating counterexample of what not to do:

      use strict; use warnings; use CGI; # create a CGI object (query) for use my $q = CGI->new; my $fh = $q->upload('file_field');

      This would be syntactic perl for backwards compatibility, but perl malpractice in 2023 for a variety of reasons. I think this tells an interesting story: CGI::Alternatives. Let's consider alternatives:

      What about something like CGI::Carp? Is this not in the contemporary core?

      I'd like to work up examples with Dancer2 and either of Miyagawa's PSGI or Plack, if anyone has anything to share from these mods.

      Personally, I prefer Mojolicious. Its tutorial includes an example, and on my scratchpad I have linked quite a few of my Mojo examples.

      I've gone with Mojo before, but I consider it the full buffet and more than I want, since I won't be using 99% of it, and deployment size and simplicity is relevant. I don't need a lot of software here. I need a few lines from the right software, and I'd like to judge by side by side comparison.

      If your ssh -T git@gitlab.com works on the same machine, then that's really strange and would indicate a problem with your network configuration or some other strange thing is going on. Though this is definitely worth debugging, I just wanted to take this oppertunity to say that I usually recommend the https protocol for all git remotes, as my experience has been it's easier to set up, especially in combination with e.g. git-credential-cache.

      Yes, so I broke through on the gitlab front at long last. (See below)

      Though I personally take a different route, probably the easiest way is to create a new project on the website, leave the option "Initialize repository with a README" or similar enabled, and then do a git clone to get that repository onto the local machine, and then start working in this new directory, pushing changes up to the server as desired

      Ok, I'll consider that approach. I'm gonna post some output, but I won't have a ginormous source post, because I have it hosted by gitlab, so I don't have to shove it code tags: yay!

      ./5.getopt.pl --infile /home/fritz/Desktop/1.hauke.txt --outfile /home/fritz/Desktop/1.hauke_trans.txt --lang DE
      /home/fritz/Desktop/1.hauke.txt, /home/fritz/Desktop/1.hauke_trans.txt, DE, 0
      
      Thank you for the comments and helpful source. I felt like a ship lost at sea without a host because I had unmoored myself from American tech giants. I tried to sign up for gitlab and showed up instead at the server that haukex's employer uses, because I'm there so often to get the details on configuring an rpi. I was refused entrance, and I thought, "gosh, I've never been refused entrance to anything in Berlin ever. Is this a new balkanization of the net?" No. It's me showing up in the wrong place, disoriented by Charybdis, blinded by Scylla....
      Ich danke Ihnen für die Kommentare und die hilfreiche Quelle. Ich fühlte mich wie ein Schiff, das ohne Host auf See verloren gegangen ist, weil ich mich von den amerikanischen Tech-Giganten losgesagt hatte. Ich habe versucht, mich bei Gitlab anzumelden und bin stattdessen auf dem Server aufgetaucht, den haukex's Arbeitgeber benutzt, weil ich so oft dort bin, um die Details zur Konfiguration einer rpi zu erfahren. Mir wurde der Zutritt verweigert, und ich dachte: "Meine Güte, mir wurde in Berlin noch nie der Zutritt zu irgendetwas verweigert. Ist das eine neue Balkanisierung des Netzes?" Nein. Ich bin nur am falschen Ort aufgetaucht, verwirrt von Charybdis, geblendet von Scylla....
      ...
      So lot's of changes with hosting situations, and I found it much better to use Getopt::Long and the command line to input the data rather than have it instantiated in a sub. I'm super rusty with the module building parts that I just couldn't even start with that. Now that I have a place to host source code again, maybe this is the type of thing I should work up again.
      So viele Änderungen mit Hosting-Situationen, und ich fand es viel besser, Getopt::Long und die Befehlszeile zu verwenden, um die Daten einzugeben, anstatt es in einem Sub instanziiert haben. Ich bin super rostig mit dem Modul Gebäude Teile, die ich einfach nicht einmal mit, dass beginnen konnte. Jetzt, wo ich wieder einen Ort habe, an dem ich den Quellcode hosten kann, ist das vielleicht die Art von Sache, an der ich wieder arbeiten sollte.
      ...
      It was a comedy of many errors to migrate from github to gitlab, but I think I'm finally getting in there with a secure ssh connection. So many hosting situations have changed this year in particular. I hope no one owns stock in Twitter right now. Wouldn't it be nice if the word "tweet" were returned to the birds?
      Es war eine Komödie mit vielen Fehlern, von github zu gitlab zu migrieren, aber ich glaube, ich komme endlich mit einer sicheren ssh-Verbindung hinein. Vor allem in diesem Jahr haben sich so viele Hosting-Situationen geändert. Ich hoffe, dass im Moment niemand Aktien von Twitter besitzt. Wäre es nicht schön, wenn das Wort "tweet" zu den Vögeln zurückkehren würde?
      
      ./5.getopt.pl : done, success.
      fritz@laptop:~/Documents/repo1$ 
      
      

      Source is trans script. I also have example.config, which shows the proper form for the config (with appropriate redaction this time):

      [deepl] url=https://api-free.deepl.com/v2/translate key=redacted

      So, yay, progress with hosting but fishing for CGI alternative code.

      Gruss aus Amiland,

        What about something like CGI::Carp? Is this not in the contemporary core?
        $ corelist CGI::Carp Data for 2022-12-20 CGI::Carp was first released with perl 5.004, deprecated (will be CPAN +-only) in v5.19.7 and removed from v5.21.0 $

        So that's a pretty definitive "no".

        Note that none of the other alternatives you have mentioned are in core either, which is fine by me. Just download what you want to use and then use it - don't worry about what is bundled in.


        🦛

Re: Using perl to script an upload page [OT - no Perl]
by kcott (Archbishop) on Jan 02, 2023 at 09:22 UTC

    G'day Aldebaran,

    [OT: my response only addresses GitLab and git commands — there's no Perl components.]

    I use GitLab extensively for $work. I regularly use dozens of repos. I do all my work from the command line. I'm aware that a number of git UIs exist; perhaps other monks can tell you about those.

    Git has a lot of commands and they take many options and arguments. I'd recommend that you bookmark "Git - Reference"; use this while reading the following and for future work.

    For the following, assume I've collected all cloned repos in a 'gitdir/' directory. I've used 'git.example.com' for the base URL. The project and repo are called (unimaginatively) 'project' and 'repo', respectively.

    When starting a new project, the first thing to do is clone the repository (which you indicated that you've already created).

    $ cd gitdir $ git clone https://git.example.com/project/repo.git
    • The 'https://git.example.com/project/repo.git' is obtained from the GitLab page. There's typically a large "Clone" button which provides URLs; copy the one you want.
    • You'd normally need to provide credentials (username/password) to proceed.
    • You should get feedback as the process is happening; it may take some time depending on the size of the repo.
    • Eventually, you'll have all directories and files under "gitdir/repo/".

    You can now perform a verification and, what I always do, avoid having to continually retype my username.

    $ cd gitdir/repo $ git remote -v origin https://git.example.com/project/repo.git (fetch) origin https://git.example.com/project/repo.git (push) $ git remote set-url origin https://username@git.example.com/project/r +epo.git $ git remote -v origin https://username@git.example.com/project/repo.git (fetch) origin https://username@git.example.com/project/repo.git (push)

    You can now begin to add, modify or delete files under "gitdir/repo/". I recommend you use the status command often; it'll mostly tell you what you need to do next. When you do this, your current directory should be "gitdir/repo/", or one of its subdirectories.

    $ git status

    Common commands you'll use at this point are:

    $ git add ... $ git commit ... $ git push

    Following a push command, your changes should now be visible via the GitLab web interface.

    Those commands probably cover 90% or more of what you'll generally need. You may also want to create branches and tag releases; very briefly:

    # Create a branch $ git branch branch_name # Create a branch and switch to it $ git switch -c branch_name # Switch to an existing branch $ git switch branch_name # Publish a new branch (now appears in GitLab UI) $ git push -u origin branch_name # Tag a release (as "v1.000") $ git tag -a 'v1.000' -m '...' # Publish a new tag (now appears in GitLab UI) $ git push origin 'v1.000'

    There's a lot more to git but I think this pretty much covers the basics. Revisit the reference documentation for more.

    — Ken

Re: Using perl to script an upload page
by hippo (Bishop) on Jan 11, 2023 at 12:13 UTC
    Q2) Do we live in a post CGI-bin world? This looks pretty ancient to be worried about buffers.

    The author of that code has chosen to be worried about buffers and that is therefore his concern. He could instead have chosen to use something like CGI::UploadEasy and not given two hoots about buffers. Personally I wouldn't worry about the use of a buffer either way. I would be more concerned about not checking the result of open(), printing out plain text in a text/html MIME type response, indenting oddly/inconsistently, using an unreferenced label and some quaint old-time syntax. :-)


    🦛