Using Cartons to automate module installs

Aldebaran has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

I've presented lately with these awful "trying to do anything on windows" threads, and my results were so dismal for installing WWW::Mechanize::Chrome on windows 10, strawberry perl, I redoubled my efforts to get an ubuntu platform to develop on instead. Well, by golly, I figured out a way to make my laptop dual-boot. It took me over a hundred failures, but then I failed to fail.

Now I have a fresh debian install, and I wanted to address some ideas that have come up in other threads lately. (I've been lurking, which requires little effort. As an intermediate with more to learn than I already know, I also need to do write-ups, else stop evolving with perl.)

I will put the details of how to get a dual boot on this particular $169 laptop between readmore tags, so that that we can just move on.

I believe this manufacturer partners with windows so that it will have windows on it when you buy it. This is what it looks like from windows' perspective:


C:\Users\tblaz\Documents\evelyn>systeminfo

Host Name:                 redacted
OS Name:                   Microsoft Windows 10 Home
OS Version:                10.0.17763 N/A Build 17763
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Workstation
OS Build Type:             Multiprocessor Free
Registered Owner:          redacted
Registered Organization:
Product ID:                redacted
Original Install Date:     8/6/2019, 1:15:23 AM
System Boot Time:          1/5/2020, 7:50:07 PM
System Manufacturer:       LENOVO
System Model:              81MV
System Type:               x64-based PC
Processor(s):              1 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 142 Stepping 1
+1 GenuineIntel ~2304 Mhz
BIOS Version:              LENOVO ASCN29WW, 3/26/2019
Windows Directory:         C:\Windows
System Directory:          C:\Windows\system32
Boot Device:               \Device\HarddiskVolume1
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (UTC-08:00) Pacific Time (US & Canada)
Total Physical Memory:     3,976 MB
Available Physical Memory: 989 MB
Virtual Memory: Max Size:  9,208 MB
Virtual Memory: Available: 3,709 MB
Virtual Memory: In Use:    5,499 MB
Page File Location(s):     C:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:              redacted
Hotfix(s):                 11 Hotfix(s) Installed.
...
Network Card(s):           2 NIC(s) Installed.
                           [01]: Realtek 8821CE Wireless LAN 802.11ac 
+PCI-E NIC
                                 Connection Name: Wi-Fi
                                 DHCP Enabled:    Yes
                                 DHCP Server:     10.0.0.1
                                 IP address(es)
...
                           [02]: Bluetooth Device (Personal Area Netwo
+rk)
                                 Connection Name: Bluetooth Network Co
+nnection
                                 Status:          Media disconnected
Hyper-V Requirements:      VM Monitor Mode Extensions: Yes
                           Virtualization Enabled In Firmware: Yes
                           Second Level Address Translation: Yes
                           Data Execution Prevention Available: Yes

C:\Users\tblaz\Documents\evelyn>
[download]

The central drama is trying to get the bios to recognize the usb stick, and our friends in Redmond are always trying to make that more difficult, which I can understand from a security standpoint. As a consumer, I like to be free to install an operating system that I can develop on. It hardly feels like the same machine. Details on how to adjust window's settings here.

First, I would like to turn to haukex's response in Re: List of Perl library modules needed run scripts. As I do the basics of a perl install and installing my html template, I find myself installing the same modules over and over again. I would like to apply the virtue of laziness to this. For example, every time I boil up a perl install, I would like to install these:

CPAN
Log::Log4perl
Perl::Tidy
YAML
[download]

Q1) How do I create and roll out a Carton of these? (I have another machine with a new ubuntu install to test on.) Is a Carton the right tool?

As I look at this output during a basic install, I wonder if I want to increase the above list?

# === Test Suggests ===
# 
#     Module                    Want     Have
#     ------------------------ ----- --------
#     CPAN::Meta::Check        0.011  missing
#     CPAN::Meta::Requirements   any    2.140
#     PadWalker                  any  missing
#     Test::Tester             0.108 1.302073
# 
...
# 
# === Other Modules ===
# 
#     Module                Have
#     --------------- ----------
...
#     Module::Runtime    missing
[download]

I used a couple new tools that I found on this source to show me what I have:

$ ./1.mod.pl 
CPAN
Capture::Tiny
Config::Tiny
Email::Find
HTML::FromText
Log::Log4perl
Module::ScanDeps
Net::SFTP::Foreign
Path::Tiny
Perl
Perl::Tidy
Spiffy
Test::Base
Test::Deep
Test::FailWarnings
Test::Fatal
Test::MockRandom
Test::More::UTF8
Test::Pod
Test::Requires
Test::WarningsExtUtils::Installed
Test::YAML
Text::Diff
Text::Template
Unicode::UTF8
YAML
$ cat 1.mod.pl 
#!/usr/bin/perl -w
use ExtUtils::Installed;
my $inst    = ExtUtils::Installed->new();
my @modules = $inst->modules();
 foreach $module (@modules){
      print $module . "\n";
}
__END__ 
$
[download]

Q2) How do I know whether I have all the core modules? How do I not leave it to chance?

I also want a Carton, or equivalent bundling, for my html templating system. I used the Module::ScanDeps functionality on its primary pm's:

$ ./scandeps.pl html7.pm
'Config::Tiny'                   => '2.24',
'Email::Find::addrspec'          => '0.09',
'HTML::Entities'                 => '3.69',
'HTML::FromText'                 => '2.07',
'HTML::Parser'                   => '3.72',
'Net::SFTP::Foreign'             => '1.90',
'Net::SFTP::Foreign::Attributes' => '1.68_05',
'Net::SFTP::Foreign::Buffer'     => '1.68_05',
'Net::SFTP::Foreign::Common'     => '1.76_02',
'Net::SFTP::Foreign::Constants'  => '1.63_05',
'Net::SFTP::Foreign::Helpers'    => '1.74_06',
'Net::SFTP::Foreign::Local'      => '1.57',
'Path::Tiny'                     => '0.110',
'Text::Template'                 => '1.58',
'Unicode::UTF8'                  => '0.62',
'trans1'                         => 'undef',
'utils1'                         => 'undef',
$ ./scandeps.pl trans1.pm
'Config::Tiny'                   => '2.24',
'Net::SFTP::Foreign'             => '1.90',
'Net::SFTP::Foreign::Attributes' => '1.68_05',
'Net::SFTP::Foreign::Buffer'     => '1.68_05',
'Net::SFTP::Foreign::Common'     => '1.76_02',
'Net::SFTP::Foreign::Constants'  => '1.63_05',
'Net::SFTP::Foreign::Helpers'    => '1.74_06',
'Net::SFTP::Foreign::Local'      => '1.57',
'Path::Tiny'                     => '0.110',
'Unicode::UTF8'                  => '0.62',
$
[download]

Q3) How do I take this output and instruct another machine to install this?

Finally, I have been developing new bash aliases with help from handy bash aliases. Some of these are perl related, some not. I haven't used them all yet, and I've added '| more' to verbose ones:

alias ..='cd ..'
alias ...='cd ../../../'
alias ....='cd ../../../../'
alias .....='cd ../../../../'
alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo termin
+al || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;
+s/[;&|]\s*alert$//'\'')"'
alias cx='chmod +x '
alias dir='dir --color=auto'
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'
alias halt='sudo /sbin/halt'
alias l='ls -CF'
alias l.='ls -d .* --color=auto'
alias la='ls -A'
alias ll='ls -alF'
alias ls='ls --color=auto'
alias meminfo='free -m -l -t'
alias music='mplayer --shuffle *'
alias nplaymp3='for i in /nas/multimedia/mp3/*.mp3; do mplayer "$i"; d
+one'
alias nplayogg='for i in /nas/multimedia/ogg/*.ogg; do mplayer "$i"; d
+one'
alias nplaywave='for i in /nas/multimedia/wave/*.wav; do mplayer "$i";
+ done'
alias playavi='mplayer *.avi'
alias playmp3='for i in *.mp3; do mplayer "$i"; done'
alias playmp4='for i in *.mp4; do mplayer "$i"; done'
alias playogg='for i in *.ogg; do mplayer "$i"; done'
alias playwave='for i in *.wav; do mplayer "$i"; done'
alias poweroff='sudo /sbin/poweroff'
alias pscpu='ps auxf | sort -nr -k 3 | more'
alias pscpu10='ps auxf | sort -nr -k 3 | head -10'
alias psmem='ps auxf | sort -nr -k 4 | more'
alias psmem10='ps auxf | sort -nr -k 4 | head -10 | more'
alias pt='perltidy -i=2 -b -utf8 '
alias reboot='sudo /sbin/reboot'
alias sc='sudo cpan'
alias shutdown='sudo /sbin/shutdown'
alias update='sudo apt-get update && sudo apt-get upgrade'
alias vlc='vlc *.avi'
[download]

alias sc='sudo cpan' is already chugging away as keystroke-saver. Q4) What perl aliases do you use? What aliases in general?

I'm always humbled by the quality of responses I get if I can manage to post a succinct question. Thank you for your response.

Comment on Using Cartons to automate module installs Select or Download Code

Replies are listed 'Best First'.
Re: Using Cartons to automate module installs by haukex (Archbishop) on Jan 30, 2020 at 20:45 UTC
Q1) How do I create and roll out a Carton of these? (I have another machine with a new ubuntu install to test on.) Is a Carton the right tool? If all you care about is getting the latest version of the modules, then you don't need Carton, only App::cpanminus (`cpanm`), because once you have a cpanfile, the command `cpanm --installdeps .` will install everything listed in that file. Carton is useful if you want to install specific versions of modules, to exactly replicate a module setup on a different machine. See Carton::Doc::FAQ. Q2) How do I know whether I have all the core modules? How do I not leave it to chance? Debian should install its `perl` package by default, which includes its `perl-modules` package. If for some reason it doesn't get installed, you can always `sudo apt-get install perl`. If you're building a fresh perl with e.g. perlbrew, you should be getting all core modules by default as well. Q3) How do I take this output and instruct another machine to install this? That particular output is suited for a `Makefile.PL` format. You could of course write a quick&dirty oneliner to turn it into `cpanfile` format: `$ scandeps.pl -R script.pl \| perl -ne \ 'printf qq{requires "%s", "%s";\n}, eval'` [download] Q4) What perl aliases do you use? What aliases in general? The only one I personally have is `alias prl='perl -wMstrict -MData::Dump'`, the rest I do by hand. I also have a `~/.perltidyrc` and `~/.perlcriticrc`. Update: Shortened oneliner.	[reply] [d/l] [select]
Re: Using Cartons to automate module installs by choroba (Cardinal) on Jan 31, 2020 at 10:13 UTC
> Q2) How do I know whether I have all the core modules? How do I not leave it to chance? Why do you care? Name all the dependencies in your Makefile.PLs, or cpanfiles, including the core dependencies. > Q4) What perl aliases do you use? What aliases in general? I don't use aliases much, as they are less flexible than functions and scripts. You can find some of my tools I've carried around for years in this GitHub repo. Some of them are written in Perl (chjoin, dups, git-graphs), some are Perl related (bead, perl-etags.pl). `map{substr$_->[0],$_->[1]\|\|0,1}[\\|\|{},3],[[]],[ref qr-1,-,-1],[{}],[sub{}^ARGV,3]`	[reply] [d/l]
Re^2: Using Cartons to automate module installs by haukex (Archbishop) on Jan 31, 2020 at 16:30 UTC
Q2) How do I know whether I have all the core modules? How do I not leave it to chance? Why do you care? I suspect it's because the first paragraph of the Carton docs is: Carton only works with perl installation with the complete set of core modules. If you use perl installed by a vendor package with modules stripped from core, Carton is not expected to work correctly.	[reply]
Re^3: Using Cartons to automate module installs by Aldebaran (Curate) on Feb 04, 2020 at 17:52 UTC
I suspect it's because the first paragraph of the Carton docs is: You were exactly right, and while the absense of core modules might be problematic for a Carton, it is immaterial to a cpanfile, which, after a ton of reading about it, seems to suit my needs fine. I have been trying to implement them in varying ways and find success in a bash script that relies on `scandeps.pl` . It seems to me that it invites disaster to compose a cpanfile by hand, never mind the tedium. It has to be named 'cpanfile' for the command `sudo cpanm --installdeps .` to work. There has been a fair amount of evolution in these matters over the years, so newer sources of information are more reliable. I found these html pages helpful and up to date: specifying-dependencies-for-your-cpan-distribution and introduction-to-distribution-metadata. The bash script wraps the invocation of `scandeps.pl`, creating a directory to house the files to be scanned and the resulting `cpanfile`. I also get a "paper trail" for what happens on STDOUT with `\| tee -a "$out"` . Here is sample output: $ ./1.cpan_dir.sh mkdir: cannot create directory �logs�: File exists basename dollar sign zero is 1.cpan_dir.sh path is /logs Munged time is 02-03-2020_17-25-09 out fn is /home/hogan/Documents/hogan//logs/02-03-2020_17-25-09.log /home/hogan/Documents/hogan /home/hogan/Documents/hogan/cpan_file_dir2 /home/hogan/Documents/hogan/cpan_file_dir2/cpanfile /home/hogan/Documents/hogan/cpan_file_dir2/scandeps.pl --> Working on . Configuring /home/hogan/Documents/hogan/cpan_file_dir2 ... OK <== Installed dependencies for .. Finishing. behold your output: requires "ExtUtils::MM_AIX", "7.44"; requires "ExtUtils::MM_Any", "7.44"; requires "ExtUtils::MM_BeOS", "7.44"; requires "ExtUtils::MM_Cygwin", "7.44"; requires "ExtUtils::MM_DOS", "7.44"; requires "ExtUtils::MM_Darwin", "7.44"; requires "ExtUtils::MM_MacOS", "7.44"; requires "ExtUtils::MM_NW5", "7.44"; requires "ExtUtils::MM_OS2", "7.44"; requires "ExtUtils::MM_QNX", "7.44"; requires "ExtUtils::MM_UWIN", "7.44"; requires "ExtUtils::MM_Unix", "7.44"; requires "ExtUtils::MM_VMS", "7.44"; requires "ExtUtils::MM_VOS", "7.44"; requires "ExtUtils::MM_Win32", "7.44"; requires "ExtUtils::MM_Win95", "7.44"; requires "ExtUtils::MakeMaker", "7.44"; requires "File::Temp", "0.2309"; requires "Module::ScanDeps", "1.27"; duration=20 Mon Feb 3 17:25:29 PST 2020 $ [download] Source: #!/bin/bash # # # keep a log named by time stamp export PATH=:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/bin:/ +bin set -u app=${0##/} mkdir "logs" pathto=/logs timename=$(date +"%m-%d-%Y_%H-%M-%S") out=$HOME/Documents/hogan/$pathto/$timename.log declare -ir SUCCESS=0 declare -ir E_FATAL=1 if [ 1 -eq 1 ] ; then echo "basename dollar sign zero is" $app echo "path is" $pathto echo "Munged time is" $timename echo "out fn is $out" echo "Time is $timename " > "$out" fi # end if [ 1 -eq 0 ] pwd \| tee -a "$out" folder='cpan_file_dir2' mkdir $folder cp /home/hogan/.cpan/build/Module-ScanDeps-1.27-0/blib/script/scandeps +.pl $PWD/$folder/ cd $folder pwd \| tee -a "$out" scandeps.pl -R .pl \| perl -ne 'printf qq{requires "%s", "%s";\n}, e +val' >>cpanfile ls -d $PWD/* \| tee -a "$out" sudo cpanm --installdeps . \| tee -a "$out" echo "behold your output:" \| tee -a "$out" cat cpanfile \| tee -a "$out" gedit $out & read -n 1 echo "duration=$SECONDS" \| tee -a "$out" date \| tee -a "$out" exit $SUCCESS [download] So, yay, that I'm getting results, but I don't quite understand the line of code I'm using to get them. Can you "talk through" what happens on this line, in particular, how the %s's get populated? `scandeps.pl -R *.pl \| perl -ne 'printf qq{requires "%s", "%s";\n}, eval'` Thank you for your comments.	[reply] [d/l] [select]
Re^4: Using Cartons to automate module installs by haukex (Archbishop) on Feb 04, 2020 at 18:35 UTC
Re^4: Using Cartons to automate module installs by Anonymous Monk on Feb 05, 2020 at 01:18 UTC


Clear questions and runnable code get the best and fastest answer
	PerlMonks