Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Re: Need writes to happen immediately.

by jbert (Priest)
on Nov 27, 2007 at 10:24 UTC ( [id://653185]=note: print w/replies, xml ) Need Help??


in reply to Need writes to happen immediately.

As mentioned above, you have various levels of buffering going on.

The application may buffer data before passing it to the kernel, in order to minimise the number of write system calls it needs to do. In perl, you control this with $|, in C you'd use fflush or similar.

The kernel will buffer data above the block device, so that it can try and do larger I/O requests (and also reorder I/O requests more efficiently). You can force a flush to disk with fsync (as noted above, you can get to this from perl too. fdatasync can be a better option for some scenarios, they probably don't affect you if you're writing to a raw partition.

If you have a snazzy disk controller, that may also have a write buffer. Most disks will also normally have a write buffer.

$ sudo hdparm -i /dev/sda | grep -i writecache AdvancedPM=yes: disabled (255) WriteCache=enabled
So even after you've sync'd from the kernel to the hardware, your data may not yet be persistent (i.e. survive a power failure)..

You can probably use hdparm to turn off write cacheing (it might depend on your disk and/or controller - don't know). However, you may decide that this isn't desirable (filesystems don't normally operate in this mode, for example). Since your filesystem needs to be resistant to disk corruption anyway (disks can go bad) it may be worth the risk that the data doesn't arrive to the platters in the event of power failure.

Another alternative at the high-end is to buy a disk controller with a battery-backed write cache, to get the best of both world, RAM-speed writes-to-cache (subsequently batched out to magentic storage optimally), together with persistence in the event of power failure.

Perl is a fine language for many applications, but I wouldn't choose it to write a filesystem. Good luck, though :-)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://653185]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others chilling in the Monastery: (4)
As of 2024-03-29 12:39 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found