Beefy Boxes and Bandwidth Generously Provided by pair Networks
Welcome to the Monastery
 
PerlMonks  

Max 16384 bytes

by bloonix (Monk)
on Jul 02, 2007 at 13:32 UTC ( [id://624438]=perlquestion: print w/replies, xml ) Need Help??

bloonix has asked for the wisdom of the Perl Monks concerning the following question:

Hello Monks,

in different code I saw this

my $blksize = (stat($FH))[11] || 16384;

for the maximum bytes for sysread() or syswrite().

My question now is from where comes the value 16384? I don't understand it. Is that a maximum buffer size? Or something else?

Thanks.
Jonny

Replies are listed 'Best First'.
Re: Max 16384 bytes
by Limbic~Region (Chancellor) on Jul 02, 2007 at 13:55 UTC
    bloonix,
    First, 16384 = 16 * 1024 or 16KB. If you read stat, you will see that the call would be returning the preferred block size for filesystem IO. This appears to be setting a default if there is not one returned. Without further context, it is hard to say for sure if it actually does that. How is the variable used elsewhere in the program? 16K -> 16KB thanks to shmem.

    Cheers - L~R

Re: Max 16384 bytes
by archfool (Monk) on Jul 02, 2007 at 14:02 UTC
    It looks like they're setting a 16K read buffer size if stat doesn't return one (i.e. it returns undef or 0 for element 11).

    The more important issue is that whoever wrote this code did NOT comment their constants. Constants should always be commented so others know what they mean AND why they were chosen.

    So, unfortunately for you, you'll never know WHY they chose 16K bytes as a buffer size. =shurg= Seems arbitrary to me... or it worked for THEIR OS.

Re: Max 16384 bytes
by shmem (Chancellor) on Jul 02, 2007 at 14:03 UTC
    in different code

    Different from what? Whence comes this code, and what purpose does it fullfil?

    From stat:

    11 blksize preferred block size for file system I/O
    The value 16384 comes from left-shift, it's 1<<14 Bytes or 2**14 Bytes or 16 kB. The expression you posted means:
    If this filehandle's file system (if any) has no preference about I/O blocksize, I'll just use 16 kB.

    Could be for socket I/O - hard to tell without context.

    Note, your system is binary (hence 2) and 7 is a magic number (hence 14) :-)

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
      Okay, sorry, to less informations.

      If I try to sysread() or syswrite() more than 16k over a socket (IO::Socket::SSL) then only 16k are read or written and not more.

      So I try to find out why only 16k are written and saw this code but it did't gives me a answer.
      Maybe he meant "indifferent code"?
Re: Max 16384 bytes
by bloonix (Monk) on Jul 02, 2007 at 15:49 UTC
    Here an IO::Socket::SSL syswrite() snippet:
    my $buffer = 'x' x 100000000; my $length = length($buffer); my $offset = 0; my $i = 0; my $client = $ipc->accept(); warn "connect from: ", $client->peerhost, "\n"; warn "print $length bytes to client\n"; while ($length) { my $written = syswrite($client, $buffer, $length, $offset); die "System write error: $!\n" unless defined $written; $length -= $written; $offset += $written; $i++; } warn "Iterations: $i\n";
    Iterations: 6104
    syswrite() writes only 16k to the socket and as I googled for a reason I saw code samples with
    my $blksize = (stat($fh))[11] || 16384;
    and though it could maybe a buffer limit or something else.

    This limit only happends with IO::Socket::SSL but not with IO::Socket::INET so I was a bit confused.
      On my box, from /usr/include/openssl/ssl3.h:
      /* Due to MS stuffing up, this can change.... */ #if defined(OPENSSL_SYS_WIN16) || \ (defined(OPENSSL_SYS_MSDOS) && !defined(OPENSSL_SYS_WIN32)) #define SSL3_RT_MAX_EXTRA (14000) #else #define SSL3_RT_MAX_EXTRA (16384) #endif

      That's why... ;-)

      --shmem

      _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                    /\_¯/(q    /
      ----------------------------  \__(m.====·.(_("always off the crowd"))."·
      ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}
        Sure.... take all the mystery out of the magical number. I mean... who'd have thought to go from Perl to C code.

        Great detective work!

        (Still should have been documented in the Perl code, too. ;)

        *argh* maybe I had to take a look into it?

        thanks!
Re: Max 16384 bytes
by Fletch (Bishop) on Jul 02, 2007 at 14:31 UTC

    Of course had the author followed better code hygiene practices that would have been DEFAULT_BLOCK_SIZE and you'd have glossed right over it . . . :)

Re: Max 16384 bytes
by naikonta (Curate) on Jul 02, 2007 at 15:37 UTC
    The block size can range from 512 upwards as long as the block size is a power of 2. Some systems put the upper limit of 131072 bytes. The original author might just wanted to set some reasonable default block size, not too low and not too high. If you really ask why s/he chose that number, heck I don't know. Perhaps, if you're studying the whole context of the program you'll find out. Or, you may want to ask the author of Net::Printer. Oracle for example, by some references, uses this number as the default value for data buffering.

    Open source softwares? Share and enjoy. Make profit from them if you can. Yet, share and enjoy!

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://624438]
Approved by ww
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others pondering the Monastery: (4)
As of 2024-04-25 23:20 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found