http://qs321.pair.com?node_id=470000


in reply to Mail::Box fails miserably when trying to open 30_000 messages maildir

"30000" smells like "signed 16-bit overflow". This problem may even be in an underlying system library (because Mail::Box doesn't use XS).

I'd start by investigating which mail box format the production environment uses.

  • Comment on Re: Mail::Box fails miserably when trying to open 30_000 messages maildir

Replies are listed 'Best First'.
Re^2: Mail::Box fails miserably when trying to open 30_000 messages maildir
by monsieur_champs (Curate) on Jun 26, 2005 at 13:19 UTC

    Production uses maildir only.

    Sorry, fglock, but I can't see the point. Why this smells like an overflow? File system is out of question, 30_000 files is large but not really a problem...

      There are some numbers in computing, which are boundries, and can cause problems (sort of like that whole y2k issue).

      Near 30,000 is the number 32,768, which is 2**15. Now, you'd think to yourself, but wouldn't there be problems at 2**16, which is a nice round number in computer terms?

      Well, no, because for an integer of (x) bits, if it's signed, it ranges from (-1*(2**(x-1)-1) to 2**(x-1). So, for a 16 bit number, it goes from -32767 to 32768. If the module in question uses XS (compiled C code), it's possible that it was compiled with a 16 bit signed number in there, which will have problems if you try to deal with numbers greater than 32,768.

      If the number is exactly 30,000 or less, this probably isn't the issue. If it's over 32,768, this could be a problem.

      From looking at the docs for Mail::Box, however, it looks to be pure perl, so I don't think this is the issue in this case.

        Unless he has Mail::Box::Parser::C installed. But it looks like that is only passed a filehandle to parse.

        cp
        ----
        "Never be afraid to try something new. Remember, amateurs built the ark. Professionals built the Titanic."
      still, it could be reasonable to write simple test script to open that dir and just count:
      opendir DIR, "that-dir"; my @files = readdir DIR; # or whatever methos Mail::Box uses print "count is ", $#files+1;
      Just to check if perl's readdir is out of question on your current build (with given libc and so on)
      Well, you know what happens when you presume, right? You make a pre of su and me.