cgraf has asked for the wisdom of the Perl Monks concerning the following question:

CPAN is failing to fetch any file, as below:
I'm trying to fetch one CPAN: LWP::UserAgent loaded ok Fetching with LWP: Fetching with LWP: Fetching with Net::FTP: Couldn't fetch MIRRORED.BY from Fetching with Net::FTP Couldn't fetch MIRRORED.BY.gz from Fetching with LWP: Useless content call in void context at /Library/Perl/5.8.1/LWP/Protoc +ol/ line 398 Fetching with LWP: Fetching with Net::FTP: Couldn't fetch MIRRORED.BY from Fetching with Net::FTP
I can download files using external programs such as wget and ftp, so the network is not a problem. I have installed a latest copy of cpan, how the same problem exists.

Replies are listed 'Best First'.
Re: CPAN problems on Mac OS X
by jaco (Pilgrim) on Aug 12, 2004 at 15:32 UTC
    I have the same problem from my work network, which of course requires passive FTP. the easiest way I've
    found in working around this is by choosing CPAN mirrors which are HTTP rather the FTP. They are the end
    of the list 51,52,53 etc.

    Just update your CPAN config.
Re: CPAN problems on Mac OS X
by been42 (Curate) on Aug 12, 2004 at 13:48 UTC
    Can you provide any info on what OS version and Perl version you are using? Also, the last sentence seems to say that you were having the problem, so you installed the latest CPAN module and that didn't fix it. Is that correct?
Re: CPAN problems on Mac OS X
by elusion (Curate) on Aug 12, 2004 at 14:00 UTC
    I aways get these errors. I had them running Jaguar, and I have them running Panther. If you just let it run it should eventually succeed. Once it does, it will work for the rest of the session.
      No, no, no, no. The problem is that Mac OS X uses passive FTP by default. You have to set an environmental variable -- FTP_PASSIVE=1 to let CPAN know how to FTP properly. If you're using bash, type "export FTP_PASSIVE=1" in your shell; tcsh, type "setenv FTP_PASSIVE 1".
        Thanks everyone for the replies. I tried setting the env variable FTP_PASSIVE=1 and cpan worked straight away. Perl Monks just saved me many hours.
Re: CPAN problems on Mac OS X
by hmerrill (Friar) on Aug 12, 2004 at 14:53 UTC
    I haven't been able to find an exact answer for your question, but hopefully I can put you on the right track. I had this same problem at work when I was on RH Linux. We had a firewall in the way, and somewhere I found that the answer was configuring CPAN to use ncftp (or ncftpget?) instead of ftp. I can't remember if that means in the ".CPAN/" file you need to change the "ftp" value from "/usr/bin/ftp" to "/usr/bin/ncftp" or what. But something along those lines is what it took to get CPAN working for me.


Re: CPAN problems on Mac OS X
by perldeveloper (Scribe) on Aug 12, 2004 at 15:35 UTC
    I have a very similar problem with my CPAN on RedHat 9, that is Net::FTP fails to download from any FTP location (as if the Net::FTP module was broken). So, as soon as CPAN switches to ncftpget, the file gets downloaded properly. I'm afraid I don't know how to disable Net::FTP, but in order to add ncftpget (or ftp) to your FTP downloading alternatives, you should edit the CPAN/ file in your Perl5 installation path (on Linux, this is /usr/local/lib/perl5/5.x.y). The file contains a list of all FTP, HTTP, proxy and other downloading settings. I believe that as soon as you get one of the utilities right (like ftp), downloading off the Internet should work with CPAN.
Re: CPAN problems on Mac OS X
by danderson (Beadle) on Aug 12, 2004 at 18:27 UTC
    I have had very similar problems at work, where ppm would not work. The workaround I used was to download the files I wanted manually (the paths are pretty easy to figure out), then in ppm I added "." as a repository (ie, the current dir) and disabled the default repositories. Search and install worked fine. You'll need to get the .ppd, the .tar.gz, and the install_* file, if there is one (eg, if you wanted Crypt-SSLeay, you'd get install_ssl, Crypt-SSLeay.ppd, and Crypt-SSLeay.tar.gz from whichever repository you want)

    Of course, I'm assuming that there's a ppm for macs... possibly an incorrect assumption. I know it works under linux and solaris, so I don't see why it wouldn't work on X/jag.

    Some older ppm versions don't allow you to configure repositories, but >= v3.1 will.

    Good luck!
Re: CPAN problems on Mac OS X
by Anonymous Monk on Aug 13, 2004 at 01:03 UTC
    Passive FTP should be the default, but for some reason it is not with Net::FTP. ncftp used to default to active FTP, but as of 3.0 it came to its senses and defaulted to passive which is why it is working.

    The reason why wget works is that your OS ships with a patch (a la Red Hat) or a default configuration file (a la FreeBSD) which overrides wget's active FTP default. (Note that if passive mode is set in a system-wide configuration file such as /etc/wgetrc or /usr/local/etc/wgetrc, there is (currently, as far as I know) no way to turn it off. As this post demostrates, there is no --active-ftp flag.)

    It seems like a simple problem with a quick fix you can apply and move on. But its really a symptom of a larger problem: the growing number of second-class, unequal peers on the Internet--those that cannot accept incoming connections. NAT (or more accurately, PAT) is the worst offender.

    Of course, NAT is a necessity in today's Internet. There simply are not enough IP addresses to go around, and ISPs will charge you more for more addresses. Many people have multiple computers. But by adding multiple computers, you sacrifice inbound connections, you sacrifice the ability to connect to peers that also cannot accept inbound connections.

    Seems like a trivial problem. After all, a home user probably doesn't ever need anyone connecting to his or her computer. Except it breaks the peer-to-peer nature of the Internet. We're already seeing NAT's drastic effects on P2P software. Firewalled users simply cannot connect to other firewalled users. The consumers are shut off from each other. One small step towards a Secure Internet.

    Of course, any reasonably knowledgeable computer user would be able to work around the limitations of NAT. Especially us Perl programmers.

    I don't know what firewall you are using, but Linux has the ftp_conn and ftp_conntrack modules for permitting active FTP. OpenBSD/FreeBSD's pf can use an FTP proxy. Some SMC routers actually replace PASV in TCP streams on port 21 with P@SV, forcing active FTP. Stand-alone residential firewalls may have similar capabilities, check your router documentation.

    Once you've poked a hole through your firewall, active FTP should now work. And all the broken programs that default to active mode.

    Sure, you could switch to passive mode, and it may work well. In passive mode, the server does the bind/listen/accept (see perldoc -f bind and man 2 bind) dance. Some feel that this is suboptimal because bind "is actually easier than selecting the first unused file descriptor, but since it is not so important for web servers, some operating systems decided not to optimize it.". The FTP server gets stuck with doing all the binds in passive FTP, when the expensive syscalls could be distributed throughout each client (in active FTP mode). But of course, the server still has to choose a local port in active mode, so I don't know how much truth there is to the claim that active FTP is superior, performance-wise.

    However, when mirroring a large FTP server using wget, I've noticed that passive mode sometimes gives me errors, well into the mirroring process, after thousands of files have been downloaded. I can't recall the exact error, but I think it was about no ports left. The transfers then fail. I've never had this problem with active FTP.

    For what its worth, in my experience active has proven far more reliable. So you might consider setting up your firewall to support active FTP if you encounter any trouble with passive. As a side-effect, Net::FTP will then begin to work out of the box. Though for most of us, switching to passive FTP as described by other replies here is the best option. But I hope this reply was useful for those who want to know a little more about this situation.


Re: CPAN problems on Mac OS X
by Anonymous Monk on Aug 12, 2004 at 20:01 UTC
    This may sound silly but are you running sudo before you run perl -MCPAN -e -shell. I had this problem and I was getting the same errors. Then I ran 'sudo perl -MCPAN -e -shell' and all was well.