Beefy Boxes and Bandwidth Generously Provided by pair Networks
There's more than one way to do things
 
PerlMonks  

Using Perl for directory listing

by htmanning (Friar)
on Sep 30, 2022 at 06:03 UTC ( #11147164=perlquestion: print w/replies, xml ) Need Help??

htmanning has asked for the wisdom of the Perl Monks concerning the following question:

Monks,

I have a large directory called registrations with a folder for each user who has registered. I need to be able to look into each of these folders via a browser, but I have restricted indexes. I thought I could use Perl to do it. I'm using File::Fine::Rule like this:

use File::Find::Rule; my $base_dir = "/usr/home/username/public_html/registrations/$dir"; my $find_rule = File::Find::Rule->new; foreach $item (@files) { $item =~ s/\/usr\/home\/username\/public_html\/registrations\/$dir +\///g; print "<a href=\"https://www.domain.com/registrations/$dir/$item\" +>$item</a><br>"; }
I feed it the directory via the URL and It works, but it does not show file sizes or any other info besides the filename. Is there another module I should be using or a better way to do it?

Thanks.

Replies are listed 'Best First'.
Re: Using Perl for directory listing
by choroba (Cardinal) on Sep 30, 2022 at 07:09 UTC
    If you have the file's path, you can use -s to get its size:
    my $size = -s $item;

    If you need more details, use stat.

    map{substr$_->[0],$_->[1]||0,1}[\*||{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^*ARGV,3]
Re: Using Perl for directory listing
by hippo (Bishop) on Sep 30, 2022 at 08:35 UTC
    Is there another module I should be using

    It depends how easy you want to make it. I like Path::Tiny for things like this. It has both size and size_human methods for displaying the size and stat for providing a more intuitive interface to the results from the system stat call which choroba has already mentioned.


    🦛

Re: Using Perl for directory listing
by Discipulus (Canon) on Sep 30, 2022 at 09:45 UTC
    Hello htmanning,

    > I have a large directory..

    just a consideration: to compute stats for large directory is an expensive operation and I'd avoid to call it on the fly on a high traffic server.

    More: the path you posted suggests a shared hosting scenario and if so most of the directories dont change in size frequently, if they do at all, so you will waste lot of disk access time to get small or no changes.

    Depending on how much critical your needs are I'd follow one these options

    A - put a cron scheduled during low traffic hours with a decent nice if needed to populate a static index with an header stating the time of the computation.

    B - create a little demon to watch the main path (or the list of all subdirs) perhaps using File::ChangeNotify or similiar modules, and just recompute the size of the subdirectory modified.

    PS rereading your post more carefully it seems you need to inspect just a subdirectory not the whole tree :) but, depending on how big and nested these subdirectoris are you may consider my approach anyway. For sure will be more fun to program it :P

    If your are maintaining a shared hosting the demon approach has other pros: you can log size changes to spot huge and unwanted uploads, having more control in realtime on quotas. If you populate a static file or a database with your information realtime you will be able to query it or to navigate it in the way you need.

    L*

    There are no rules, there are no thumbs..
    Reinvent the wheel, then learn The Wheel; may be one day you reinvent one of THE WHEELS.

      Another option would be to regularly load the relevant data into a proper database with a background worker, then the frontend can access this data in milliseconds.

      PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP
Re: Using Perl for directory listing
by LanX (Sage) on Sep 30, 2022 at 09:24 UTC
    > show file sizes or any other info

    I think you want -X file test functions

    -r File is readable by effective uid/gid. -w File is writable by effective uid/gid. -x File is executable by effective uid/gid. -o File is owned by effective uid. -R File is readable by real uid/gid. -W File is writable by real uid/gid. -X File is executable by real uid/gid. -O File is owned by real uid. -e File exists. -z File has zero size (is empty). -s File has nonzero size (returns size in bytes). -f File is a plain file. -d File is a directory. -l File is a symbolic link (false if symlinks aren't supported by the file system). -p File is a named pipe (FIFO), or Filehandle is a pipe. -S File is a socket. -b File is a block special file. -c File is a character special file. -t Filehandle is opened to a tty. -u File has setuid bit set. -g File has setgid bit set. -k File has sticky bit set. -T File is an ASCII or UTF-8 text file (heuristic guess). -B File is a "binary" file (opposite of -T). -M Script start time minus file modification time, in days. -A Same for access time. -C Same for inode change time (Unix, may differ for other platforms)

    Cheers Rolf
    (addicted to the Perl Programming Language :)
    Wikisyntax for the Monastery

Re: Using Perl for directory listing
by kcott (Archbishop) on Sep 30, 2022 at 20:18 UTC

    G'day htmanning,

    Here's a possible solution. It's basically just a technique; adapt to your needs.

    "I feed it the directory via the URL ..."

    That sounds like the the URL could be manipulated to inject malicious code. The following thwarts such attempts (in the example runs below I just used `pwd` to demonstrate this; that could, of course, be `rm ...`). Regardless of whether you use this, or another, solution, you should bear this possibility in mind: never trust user input as even innocent, accidental typos can cause disasters.

    #!/usr/bin/env perl use strict; use warnings; use constant { BASE_URL => 'https://www.example.com/reg/', LS => '/usr/bin/ls', LS_OPTS => [qw{-l -h}], }; use IPC::System::Simple 'capturex'; die "Usage: $0 dir\n" unless @ARGV; my $dir = $ARGV[0]; my $base_dir = '/home/ken/tmp/pm_11147164_web_dir_listing/'; my $dir_path = $base_dir . $dir; die "ERROR: User '$dir' does not exist.\n" unless -e $dir_path && -d _ +; chdir $base_dir; my @listing; for (capturex(LS, @{+LS_OPTS}, $dir_path)) { next if 0 == index $_, 'total'; /^(.*?\s)(\S+)$/; push @listing, qq{<li>$1<a href="} . BASE_URL . qq{$dir/$2">$2</a></li>\n}; } print "<p>User: <strong>$dir</strong></p>\n"; if (@listing) { print "<ul>\n"; print for @listing; print "</ul>\n"; } else { print "<p>User '$dir' has no files.</p>\n"; }

    Sample runs using the directory structure shown:

    $ ls -lR .: total 4 drwxr-xr-x 1 ken None 0 Oct 1 05:18 u1 drwxr-xr-x 1 ken None 0 Oct 1 04:00 u2 drwxr-xr-x 1 ken None 0 Oct 1 04:33 u3 -rwxr-xr-x 1 ken None 839 Oct 1 06:07 web_dir_listing.pl ./u1: total 5 -rw-r--r-- 1 ken None 2002 Oct 1 05:11 u1_a -rw-r--r-- 1 ken None 501 Oct 1 05:18 u1_b ./u2: total 4 -rw-r--r-- 1 ken None 4004 Oct 1 05:18 u2_c -rw-r--r-- 1 ken None 0 Oct 1 04:00 u2_d ./u3: total 0 $ ./web_dir_listing.pl u1 <p>User: <strong>u1</strong></p> <ul> <li>-rw-r--r-- 1 ken None 2.0K Oct 1 05:11 <a href="https://www.examp +le.com/reg/u1/u1_a">u1_a</a></li> <li>-rw-r--r-- 1 ken None 501 Oct 1 05:18 <a href="https://www.examp +le.com/reg/u1/u1_b">u1_b</a></li> </ul> $ ./web_dir_listing.pl u2 <p>User: <strong>u2</strong></p> <ul> <li>-rw-r--r-- 1 ken None 4.0K Oct 1 05:18 <a href="https://www.examp +le.com/reg/u2/u2_c">u2_c</a></li> <li>-rw-r--r-- 1 ken None 0 Oct 1 04:00 <a href="https://www.examp +le.com/reg/u2/u2_d">u2_d</a></li> </ul> $ ./web_dir_listing.pl u3 <p>User: <strong>u3</strong></p> <p>User 'u3' has no files.</p> $ ./web_dir_listing.pl u4 ERROR: User 'u4' does not exist. $ ./web_dir_listing.pl 'u1;pwd' ERROR: User 'u1;pwd' does not exist. $ ./web_dir_listing.pl Usage: ./web_dir_listing.pl dir

    See also: IPC::System::Simple (noting that capturex() does not invoke the shell).

    — Ken

Re: Using Perl for directory listing
by cavac (Vicar) on Oct 03, 2022 at 08:12 UTC

    I feed it the directory via the URL and It works

    Are you sure? Do you validate the directory path passed in via the URL? Without more info, this sounds awfully like a potential directory traversal exploit to me.

    PerlMonks XP is useless? Not anymore: XPD - Do more with your PerlMonks XP

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://11147164]
Approved by choroba
Front-paged by kcott
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others making s'mores by the fire in the courtyard of the Monastery: (1)
As of 2023-05-29 03:01 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found

    Notices?