http://qs321.pair.com?node_id=11122650

Lady_Aleena has asked for the wisdom of the Perl Monks concerning the following question:

Several months ago I read a tutorial on module creation. It got me thinking about releasing some of my modules. I got to work getting my code organized. At the time I had all of my work in the directory for my site. So I moved my general purpose modules to their own directory and then started reading more about what is needed to get a module published on CPAN.

I first installed Module::Starter. It seemed like a good place to start, but then Dist::Zilla was suggested, so I installed it. Most recently Minilla suggested, and now it is installed. The problem is, I do not know which one to use. Do I use any of those at all, or is there yet another packaging module (with executable) out there?

Module::Starter (module-starter) appears to be for those starting with well thought out plans for what they want to write. I have been told Dist::Zilla (dzil) is not for a first time releaser. Minilla appears to be for those who have a complete package written and just need to get it compressed to send to CPAN.

I have also read Release::Checklist.

I know after the code, documentation, and tests are written; the rest should take maybe half an hour or less. I have lost all confidence I had several months ago when I thought it would be easy. I am drowning in an ocean of documentation that most might find to be only a wading pool. I am reaching out and grasping at whatever scrap I can find that might match my specific circumstances, but I keep getting bogged down. I am fairly sure releasing modules to CPAN should not cause tearful emotional breakdowns on a semi-regular basis, especially when it is just one module with less than 30 lines of code to it.

My circumstance is I have around 60 free floating .pm files I am thinking about publishing to CPAN. All have plain old documentation in various states of completion. All have at least their first tests written. (Only 2 have more than the use_ok test in a BEGIN block.)

The module I want to release first is Fancy::Open. It has a separate .pod file and one test file (see below).

Where do I go from here? I am willing to do this all "by-hand" if need be since this is my very first package and release.

Apologies if this is not well written. I am so confused, but I do not want to give up like I have on other aspects of Perl.

Note: Due to calls for a name change, I moved Fancy::Open to File::Slurp::Affix (.pod - test file).

My OS is Debian 10 (Buster); my perl versions are 5.28.1 local and 5.16.3 or 5.30.0 on web host depending on the shebang.

No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!
Lady Aleena

Replies are listed 'Best First'.
Re: What do I use to release a module to CPAN for the first time?
by tobyink (Canon) on Oct 10, 2020 at 00:02 UTC

    Minilla is probably a very good place to start.

    Use minil new to create a new distribution in a blank new directory called "Fancy-Open". See the getting started section of the documentation.

    Then copy your existing module, pod, and tests, overwriting the ones which Minilla created for you. Edit the "cpanfile" file to show your project's dependencies.

    That should be about it.

    It is an easy and fast process once you get the hang of it, but releasing a distribution which will actually upload okay and people will be able to install without issues is a STEEP learning curve to start off. If you upload a file that gets loads of fails on CPAN testers, don't panic — you can just bump the version number and upload again. Just keep coming back for advice and you'll get there!

Re: What do I use to release a module to CPAN for the first time?
by davido (Cardinal) on Oct 10, 2020 at 20:23 UTC

    The following is one of many ways to do it. It happens to be a pretty straightforward way. I don't recommend getting mired in other alternatives until you understand every step in this example.

    1. Understand the shape of a distribution. A distribution lives within a filesystem tree that looks like this:

      tree Foo-Bar/ Foo-Bar/ Changes .gitignore lib Foo/ Bar.pm LICENSE Makefile.PL MANIFEST README t/ 00-load.t manifest.t pod-coverage.t pod.t xt/ boilerplate.t

      The names of some of the files will differ. And in your final, shippable distribution, there will be some additional files not shown here. And there are some files above that you won't ship in the distribution, but will put into your GitHub repo (or maybe both if you want). We'll get to that in a minute.

      The crux is you need a Makefile.pl, a MANIFEST, LICENSE, README, Changes, lib/ (directory containing your module code), and t/ (directory containing your tests). "Need" is a little bit of a white lie, because you can ship a distribution without some of these files, but this is a really good starting point.

    2. Generate your distribution framework. You can do this by hand, or by copying some other similarly laid out distribution, or by using a tool such as Module::Starter, which installs a module-starter command. Let's do that:

      module-starter --author='David Oswald' --email='xxxxxx@cpan.org' --module='Foo::Bar' --eumm --license=artistic2 --genlicense --verbose --ignores=git

      This produces the following output on your display:

      Created Foo-Bar Created Foo-Bar/lib/Foo Created Foo-Bar/lib/Foo/Bar.pm Created Foo-Bar/t Created Foo-Bar/t/manifest.t Created Foo-Bar/t/pod.t Created Foo-Bar/t/00-load.t Created Foo-Bar/t/pod-coverage.t Created Foo-Bar/xt Created Foo-Bar/xt/boilerplate.t Created Foo-Bar/.gitignore Created Foo-Bar/Makefile.PL Created Foo-Bar/Changes Created Foo-Bar/README Created Foo-Bar/LICENSE Added to MANIFEST: Changes Added to MANIFEST: lib/Foo/Bar.pm Added to MANIFEST: LICENSE Added to MANIFEST: Makefile.PL Added to MANIFEST: MANIFEST Added to MANIFEST: README Added to MANIFEST: t/00-load.t Added to MANIFEST: t/manifest.t Added to MANIFEST: t/pod-coverage.t Added to MANIFEST: t/pod.t Added to MANIFEST: xt/boilerplate.t Created Foo-Bar/MANIFEST Created starter directories and files

      If you look closely, you'll see that matches the tree output I showed earlier. It created everything in the distribution tree for you. You are now done with Module::Starter. That's the extent of its involvement. If you were using Dist::Zilla (which you shouldn't be using for your first module, or maybe even your first ten modules), you would continue using it past this point. But we're using Module::Starter in this explanation, and it has finished its work.

    3. Add your module code into lib/. Hopefully your module code follows the naming convention that you used when invoking Module::Starter. Otherwise, just start over.

    4. Now you add your code tests to t/, and your author tests to xt/. While you're at it, you investigate the tests that were included automatically by Module::Starter. Some of them you may want to keep, some you may want to remove.

    5. Next you add the following two lines to your module:

      our $VERSION = '0.01'; $VERSION = eval $VERSION;

      The second line is mostly useful if you intend to use the underscore in version numbers to designate a CPAN upload as being a testing release only; '0.01_01' for example, containing an underscore, would not be indexed by the PAUSE indexer, so people wouldn't download it by mistake. Worry about that later. Read perlmodstyle for a better understanding of this.

      Remember that any time you release a module to CPAN you need to bump the version number by one.

    6. Now you run all your tests and get them passing: prove -lv

    7. Next edit Makefile.PL to add your module's prerequisites unto the PREREQ_PM hash. Add your testing prerequisites into the TEST_REQUIRES hash, and any build prerequisites (which you shouldn't need to worry about) into the CONFIGURE_REQUIRES hash. Set your MIN_PERL_VERSION appropriately. Verify everything else in Makefile.PL looks right.

    8. Add all files that you have added to your distribution manually to MANIFEST

    9. Edit README to your liking.

    10. Edit Changes to reflect the initial release of the module. See examples of how others set up their Changes file, and pick a format you like.

    11. Make your distribution:

      perl Makefile.PL make make test

      Now you have verified that the distribution, as built into a temporary blib/ folder, still passes its tests.

    12. Build a test distribution:

      make disttest
    13. Copy META.json and META.yaml out of your test distribution path (something like Foo-Bar-0.01/META.json) into your top level directory. Then remove the Foo-Bar-0.01/ directory, and add the two META.* files to your MANIFEST.

    14. Optionally, if you think you may have missed something, make manifest, though it might be worth backing up the original MANIFEST first and comparing them afterwards.

    15. Now you can make your distribution:

      make dist

      At this point you'll have a tarball that you can upload to CPAN, named something like Foo-Bar-0.01.tar.gz

    16. Upload your distribution through PAUSE. Watch the CPAN testers after a few hours to see if there are any test failures. This is why I like to start with a "development release" (a release that has an underscore in the version number); that way nobody starts downloading my module on accident before it's ready. Once you're happy with the test results, bump your version number to a non-development number, and repeat all the steps to testing and releasing the new version. This includes copying down a new version of META.* from the make disttest output.

    That's it. You've spent an hour or two your first time through it, and made a few mistakes, but it's ok because you used a developer's release version number at first, and you can always delete uploads through PAUSE. Optionally, but highly encouraged, would be to put your distribution on GitHub, and to add links to the repo in the meta hash within Makefile.PL There are plenty of examples of people doing this on CPAN, so you can do some research and see how it's done. It's easy.

    There have been a lot of details glossed over here; things that maybe can be done more optimally, or automated even more, but this is your first module. You should be keeping it simple at first. We could have used Dist::Zilla, and we could have used Module::Build, or another framework. But the way I enumerated here is pretty straightforward, and commonly used.

    I consider the following to be required reading: ExtUtils::MakeMaker, ExtUtils::MakeMaker::Tutorial, ExtUtils::MakeMaker::FAQ, Module::Starter and perldoc module-starter, and perlmodstyle. There are a lot of other things you can read, but these are good starting points. Ignore most other options for now. Your first release is hard enough that you don't need to get distracted by a bunch of other ways to do it. You want one way; a way that allows you to see what's going on so that you can learn how the other ways streamline the process (if you can call it that; I still use the basic "by hand" approach outlined here).

    One last point is whether it makes sense to upload anything at all. I would opt for keeping the distribution small; make several small stand-alone distributions rather than one kitchen sink distribution. Ask, are others going to use this? If you believe they are, upload it. If you believe only you will use it, don't. People are more likely to use small single-purpose modules that do one thing well than to use a module that provides everything you've ever written, the sum of which is only useful to your own project. But definitely, if you're providing utilities that are otherwise absent from CPAN, or that you're doing better than what's already up there, or filling a use case that isn't ideally covered already, upload.

    Make this experience pleasant by taking your time to understand the process. You may be feeling the pressure of trying to rush through the learning curve. Nobody's life depends on your module getting onto CPAN, so realize that it's ok to take as long as you want, or even to never upload it. But if you do want to do this, understand that it's not only possible, but also well documented. People with less experience than you have managed it with less aptitude for it than you have. You'll get it, too. The book Intermediate Perl, 2nd Edition is sort of what I think you need for that next step in your journey with Perl and the Perl ecosystem. It contains a chapter on contributing to CPAN, that explains PAUSE, the indexer, the basic components of a CPAN distribution, and how to build one. The reading I linked to above might be "next steps" reading after spending an hour with this chapter in Intermediate Perl. But this book is good for other reasons too. I recommend it for you; the whole book. It is unfortunate that a portion of the chapter on contributing to CPAN starts you down the Build.PL path instead of the ExtUtils::MakeMaker / Makefile.PL path. It's not that Build.PL isn't great; I just wish that we could agree on teaching one way first, and by popularity the EU::MM / Makefile.PL way is the most common. Nevertheless, the explanations in the book are useful.


    Dave

      I got to step 14, but I have a mess in the directory. What do I delete and what do I keep? I uploaded the entire mess to its new repository.

      My OS is Debian 10 (Buster); my perl versions are 5.28.1 local and 5.16.3 or 5.30.0 on web host depending on the shebang.

      No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!
      Lady Aleena

        blib/ and its subdirs are "build lib", and are throwaway. make clean will remove blib/, and will remove a few other things.

        pm_to_blib is also build output, and goes away with make clean.

        File-Slurp-Affix-0.01_01/ and its subdirs are the output of make disttest. After you've pulled out the META.* files that it provides, you can safely delete it.

        Your .gitignore should also be configured to ignore most of the generated files, except the META.yaml and META.json files that you'll pull into the top level. And if the make distcheck command is reporting files that you don't want to add to your distribution, you can set up a custom MANIFEST.SKIP (but will need to add to it the defaults that you are getting for free when that file doesn't exist).

        Don't bundle MYMETA.* with the distribution. Those are generated just in time. But META.* are useful to the CPAN indexers. I think the man\d/ directories are generated and will be removed by make clean, or make realclean, so don't bundle them.


        Dave

Re: What do I use to release a module to CPAN for the first time?
by hippo (Bishop) on Oct 10, 2020 at 10:07 UTC
    I am fairly sure releasing modules to CPAN should not cause tearful emotional breakdowns on a semi-regular basis, especially when it is just one module with less than 30 lines of code to it.

    Indeed not. While you could use one of the frameworks for release, I would not necessarily recommend that for your first go. Once you know how it is all done then you can look to automate away the boring parts. But in essence it is this simple (assuming you have used ExtUtils::MakeMaker):

    1. Before release: make sure all your tests pass, check everything in to your VCS and (optionally) tag it.
    2. make distclean - this also does a distcheck and if it reports any problems, address those before continuing.
    3. perl Makefile.PL
    4. make tardist

    You now have a tarball which you can upload to PAUSE.

    The module I want to release first is Fancy::Open.

    Have you read On The Naming of Modules? I suggest you do. Among much other good advice it says this:

    Even though the module naming is in practice a first-come first-served process, it is quite impolite to grab top-level names. Yes, even if your project/product is named with just a single word, please think of people trying to find something that would help them in their problems. Unless they know of your project/product, they might not ever find your module.

    Remember that though you may be the first to contribute to a namespace, you may well not be the last or the only one. Someone might later want to use the namespace, for something unrelated to your modules.

    So perhaps consider a different name. Did you post to PrePAN?

    The doc goes on to say:

    The modules@perl.org (the mailing list for PAUSE admins) and module-authors@perl.org can help you choose a good name. Not only are they generally good at names, but they also know quite a bit about what is already on CPAN. They can help you choose a name that puts your module into the right place with all of the other modules.

    I hope this is useful to you.


    🦛

      I second this advice on naming. The proposed name tells users nothing in relation to what the module does, and 'Fancy' is subjective.

        A subjective name is not necessarily a problem —XML::Simple is the most complex XML parser I know— but the descriptor shouldn't be the top level! There's already File:: which contains similar module File::Slurper. How about File::FancySlurp?

      I have never used ExtUtils::MakeMaker before. I had heard of it in passing but had not looked into it until you posted about it. Right now I have not even created the distribution directory yet.

      As for naming, I wish I did not have to use the word "slurp". I have an irrational bias against it, but I will probably cave and use it since it appears to be the common way to describe what this module does. I posted on PrePAN and got the suggestion of File::Slurp::Affix, which would make the subroutine name slurp_affix. I will miss the word Fancy. However, I guess I should not introduce a new namespace with my first module release.

      My OS is Debian 10 (Buster); my perl versions are 5.28.1 local and 5.16.3 or 5.30.0 on web host depending on the shebang.

      No matter how hysterical I get, my problems are not time sensitive. So, relax, have a cookie, and a very nice day!
      Lady Aleena
Re: What do I use to release a module to CPAN for the first time?
by tinita (Parson) on Oct 10, 2020 at 00:16 UTC
    Apart from choosing which tool to use for generating a distribution/release (I like Dist::Zilla, the only downside IMHO are the many dependencies), you might also want to look at my checklist which covers various aspects: https://github.com/perlpunk/perl5-module-meta
    I collected these things when I started maintaining the perl module repository for openSUSE, and every point on the checklist has its reason. I've seen many things ;-)
    Feedback welcome!
Re: What do I use to release a module to CPAN for the first time?
by pryrt (Abbot) on Oct 10, 2020 at 17:32 UTC

    So, when I started down the CPAN-author path, it was with Games::Literati, which I was taking over from the previous author. At the time I inherited it, the distribution was as simple as shown here in my github history -- the module file, the Makefile.PL, a README, a Changes file, and a test.pl; the Makefile.PL (which is what enables you to do the make distclean; perl Makefile.PL; make tardist that ++hippo recommended) was extremely minimalist, but got the job done. My first v0.031 release (in github here) wasn't much more complicated: I just added a couple of extra features to the Makefile.PL. Over time, as I dealt with that and my other distros that I started creating, I learned more about ExtUtils::MakeMaker features, and found a release workflow that worked better for me, so now my Makefile.PL is rather more complicated... but the simple versions are sufficient to get the job done.

    I highly recommend for just getting started that you start simple, and don't expect to get it perfect the first time -- it will likely be good enough, and you can make incremental improvement in your process as time moves on. The only way you're going to find a way that makes you happy is to start somewhere, and find what you like and don't like about the process.

    Regarding naming: I concur with ++hippo and ++marto that creating a new Fancy:: top-level name isn't the best. Given your description of the Fancy:: modules, they appear to be wrappers for builtin functions, but that add or incorporate features that you find lacking in the builtin versions. I can see why you would desire to group them all together under a name, but maybe something::Open -- where something:: would be wherever other wrappers of low-level functions go (but since I've never looked for such, I wouldn't know what that something:: should be, sorry -- it might be more than one level, and you might be able to use ::Fancy:: as an intermediate if the something:: is descriptive enough, otherwise maybe use ::BuiltinWrapper::Fancy:: or something that has more meaning behind it than just "Fancy"). I know the official advice is to go to PrePAN and/or the module-authors list for naming recommendations... but it seems to me to be a hassle to create yet another account in yet another location, when Perlmonks already has some of the best of the Perl community, and someone here should be able to have better suggestions for naming than I've come up with.

Re: What do I use to release a module to CPAN for the first time?
by ikegami (Patriarch) on Oct 11, 2020 at 05:06 UTC

    Tip:

    $opt->{'encoding'} ? $opt->{'encoding'} : 'utf-8'
    is more idiomatically/simply written as
    $opt->{encoding} || 'utf-8'

    Tip:

    if ( length($line) > 0 || $opt->{'empty'} && $opt->{'empty'} eq 'fill' + ) { $final_line = $prefix . $line . $suffix; } elsif ( $opt->{'empty'} && $opt->{'empty'} eq 'blank' ) { $final_line = ''; } elsif ( $opt->{'empty'} && $opt->{'empty'} eq 'undefined' ) { $final_line = undef; } else { next; }
    could be simplified to
    my $empty = $opt->{empty} || ''; ... if ( length($line) || $empty eq 'fill' ) { $final_line = $prefix . $line . $suffix; } elsif ( $empty eq 'blank' ) { $final_line = ''; } elsif ( $empty eq 'undefined' ) { $final_line = undef; } else { next; }

    But I'd personally setup some variables beforehand.

    my $empty = $opt->{empty} || ''; my $trim_empty = !$empty; my $keep_empty = $empty eq 'fill'; my $empty_replacement = $empty eq 'blank' ? '' : undef; ... if ( length($line) || $keep_empty ) { push @array, $prefix . $line . $suffix; } elsif ( !$trim_empty ) { push @array, $empty_replacement; }
Re: What do I use to release a module to CPAN for the first time?
by perlfan (Vicar) on Oct 10, 2020 at 01:12 UTC
    Dist::Zilla has been the guilty pleasure (at least for me) in terms of releasing to CPAN. Before I used that, I did it the manual way, which is not terrible but can be tedious. So I recommend that or this new thing that's come up recently, Minilla. Both mentioned already in this thread.

    As long as you can afford to crap up your working environment, I recommend DZ. But I use it typically inside of a perlbrew environment where I can isolate all of the tons and tons (and TONs) of stuff you need to manage your distro using a hand few of commands and a dist.ini file. That said, next time I roll something I might try Minilla.

      Dist::Zilla for most people is overkill IMHO, note the docs mention 'ludicrous demands in terms of prerequisites.', combined with a relatively steep learning curve, likely to make this daunting for OP .

        That's exactly why I put that shit in an isolated environment. For me it's often worth it to wait the 30 minutes to install xD... but no, this would totally blow out any respectible work space. This comes into perspective when one realizes that it's not a prereq for your module, it's a build/dev environment. I am sure you know that, but that is fuzzy for most (was for me, anyway).
Re: What do I use to release a module to CPAN for the first time?
by Anonymous Monk on Oct 10, 2020 at 16:42 UTC

    I always recommend Minilla to new authors and those who don't want to have to think about their authoring tool. As others have covered it is very good at "just doing it" for the 90% case.

    Another simple tool I recommend to new authors if they are the type that wants to have more control over the process is mbtiny. This is a bit limited since it can only use Module::Build::Tiny as the installer, but for the cases it works for, it makes the release process very simple and leaves everything else up to you.

    If you want full control of the process, have complex needs, or just are very bored, then it may be worth diving into Dist::Zilla. I encourage starting with Dist::Zilla::Starter, and then using @Starter::Git (or @Starter if you are not using git) (shameless plug, I've written all of these). Similarly to mbtiny, it will simplify all of the core processes for you in a modern way, and let you decide how everything else should work via configuration and additional plugins. I also encourage anyone running into trouble with Dist::Zilla to join #distzilla on irc.perl.org where we are happy to help with walking through or debugging any variety of issue.

    -Dan

Re: What do I use to release a module to CPAN for the first time?
by Anonymous Monk on Oct 16, 2020 at 09:08 UTC
    I can't see anyone but you using this. What motivation to put this on CPAN?