Re^3: Perl on Windows Best Practices?

SMB/My Network Places/CIFS handshaking/latency is slower than any HD IMHO or even HTTP. Look at it in wireshark one day, many handshake back and forth set up stuff, lots of metadata queries on the files, attribute reading and setting, file pointer setting, etc. Dozens or hundreds of sub 100KB files gets slow and will not saturate a network connection mbps wise in my experience between Windows computers. Of course copying files with Explorer and Perl interp starting up and loading PMs are both synchronous I/O. I tried copying my perl dir from 1 XP machine to a Server 2003 machine with Explorer, no HD light flashing on the source machine (the whole perl dir was in read cache from a recent grep of the dir), probably none on the Server 2003 machine (8 GB of RAM, 7 GB probably free), the average was 1-2 mbps up (to dest) and 500 kbps down (from dest) when copying .pl and .pm files, and jumped to 70 mbps and 500 kbps down (from dest) when copying .pdb files. I have a 100mbps ethernet network. I dont think file caching works over SMB connections, atleast with NT 5, since multiple clients or the server itself can read write the file at any time and SMB has no way of pushing cache invalidation messages across the wire or doesn't use them in most cases.

It seems things improved with NT 6, http://en.wikipedia.org/wiki/Server_Message_Block#SMB2. I have no NT 6 machines.

Comment on Re^3: Perl on Windows Best Practices?

Replies are listed 'Best First'.
Re^4: Perl on Windows Best Practices? by BrowserUk (Patriarch) on Jul 31, 2012 at 04:05 UTC
I dont think file caching works over SMB connections, Hm. l: is my net use for live.sysinternals.com\tools via my (currently) 700kbs broadband and wifi: `C:\test>net use l: Local name l: Remote name \\live.sysinternals.com\tools Resource type Disk The command completed successfully.` [download] The first time I download procexp.exe it takes 34 seconds for the 2.5 MB file: `C:\test> prompt [$t] $p$g [ 4:41:09.11] C:\test>copy l:\procexp.exe . 1 file(s) copied. [ 4:41:43.33] C:\test>dir procexp.exe 11/07/2012 17:38 2,691,192 procexp.exe 1 File(s) 2,691,192 bytes 0 Dir(s) 101,746,388,992 bytes free` [download] After that it is almost instantaneous. This is done by hitting cursor up and enter as fast as I can: `[ 4:56:14.29] C:\test>del procexp.exe && copy l:\procexp.exe . 1 file(s) copied. [ 4:56:14.34] C:\test>del procexp.exe && copy l:\procexp.exe . 1 file(s) copied. [ 4:56:14.38] C:\test>del procexp.exe && copy l:\procexp.exe . 1 file(s) copied. [ 4:56:14.42] C:\test>del procexp.exe && copy l:\procexp.exe . 1 file(s) copied. [ 4:56:14.47] C:\test>del procexp.exe && copy l:\procexp.exe . 1 file(s) copied.` [download] Between .04 and .05 of a second. That can only come from local caching. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply] [d/l] [select]
Re^5: Perl on Windows Best Practices? by bulk88 (Priest) on Jul 31, 2012 at 05:17 UTC
I can't replicate that because of my residential cable ISP. No. Time Source Destination Protoc +ol Info 3 12.791678 192.168.1.112 157.56.203.150 ICMP + Echo (ping) request 4 14.956316 192.168.1.112 157.56.203.150 ICMP + Echo (ping) request 5 17.456431 192.168.1.112 157.56.203.150 TCP + 11975 > microsoft-ds [SYN] Seq=0 Win=16384 Len=0 MSS=1460 6 17.456837 192.168.1.112 157.56.203.150 ICMP + Echo (ping) request 7 17.465186 10.240.176.53 192.168.1.112 ICMP + Destination unreachable (Communication administratively filtered) 8 19.956352 192.168.1.112 157.56.203.150 ICMP + Echo (ping) request 9 20.441617 192.168.1.112 157.56.203.150 TCP + 11975 > microsoft-ds [SYN] Seq=0 Win=16384 Len=0 MSS=1460 13 22.456477 192.168.1.112 157.56.203.150 TCP + 11976 > netbios-ssn [SYN] Seq=0 Win=16384 Len=0 MSS=1460 16 22.465609 10.240.176.53 192.168.1.112 ICMP + Destination unreachable (Communication administratively filtered) 17 25.371335 192.168.1.112 157.56.203.150 TCP + 11976 > netbios-ssn [SYN] Seq=0 Win=16384 Len=0 MSS=1460 18 25.379983 10.240.176.53 192.168.1.112 ICMP + Destination unreachable (Communication administratively filtered) 19 26.376227 192.168.1.112 157.56.203.150 TCP + 11975 > microsoft-ds [SYN] Seq=0 Win=16384 Len=0 MSS=1460 20 31.405568 192.168.1.112 157.56.203.150 TCP + 11976 > netbios-ssn [SYN] Seq=0 Win=16384 Len=0 MSS=1460 [download] did a "net use \\live.sysinternals.com\tools" to generate the above. console says `System error 67 has occurred. The network name cannot be found.` [download] A residential ISP can't be blamed for blocking SMB ports. update: Are you sure you are using SMB over the internet and not WebDAV? I turned on my Web Client service on XP, and retried to map the drive letter. I then ran `use Win32API::File 'QueryDosDevice'; print $olTargetLen = QueryDosDevice( 'W:', $osTargetPath, [] ); print "\n\n"; print $osTargetPath;` [download] and got `74 \Device\WebDavRedirector\;W:000000000000fce5\live.sysinternals.com\too +ls` [download] Thats not the SMB driver. If I rerun the script on an actual local network SMB share I get `54 \Device\LanmanRedirector\;Z:000000000000fce5\removedservername\c` [download] and LanmanRedirector is the correct driver on NT for SMB.	[reply] [d/l] [select]
Re^6: Perl on Windows Best Practices? by BrowserUk (Patriarch) on Jul 31, 2012 at 06:12 UTC
Are you sure you are using SMB over the internet and not WebDAV? Yes, I used WebDAV for the demo. I don't have an accessible other local machine right now. My other machine is taking a sabbatical. But I'm pretty sure that file shared across the local LAN were cached locally going right back to NT4. The first connections were always slow; and directory listings were never cached back then, so they were also slow; but individual file metadata was cached and reopening a remote file was always quicker the second time around. You are right about Vista/SMB2 though. Reading around, it seems to have improved things a lot through the use of bigger reads; coalesced commands and overlapped IO. With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday' Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error. "Science is about questioning the status quo. Questioning authority". In the absence of evidence, opinion is indistinguishable from prejudice. The start of some sanity?	[reply]


XP is just a number
	PerlMonks