Re^13: Strange memory leak using just threads (forks.pm)

Use $#array and a 4k page of COW'd memory get copied...and (at least) another 4k gets copied

This is only true if all these variables reside in different memory pages, otherwise just one page is copied (and afterwards, the copy is modified). As such, this example of events is misleading.

These piecemeal requests for another single page of VM followed by a 4k copy through ring 0 add up to far more than an up-front single request for as many pages of VM as are required followed by a single ring 3 rep mov.

Umm, when put as the general case, no. Moving stuff around in RAM is expensive. The CPU/MMU operations required to throw a page fault and allocate memory are insignificant compared to the time it takes to actually copy the data in memory. Which means that there is only a very slight difference (over the lifetime of a process) between copying the whole process address space in one move or doing so whenever a page is dirtied.

Also, for most use cases, the process will not have to read/write access its entire memory throughout its lifetime. For the far more common usage scenario, when part of the memory remains untouched and therefore shared, copy-on-write is far more efficient than copy-all-at-once (even only talking about speed, this is obviously even more true for process memory consumption).

All dogma is stupid.

Comment on Re^13: Strange memory leak using just threads (forks.pm) Download Code

Replies are listed 'Best First'.

Re^14: Strange memory leak using just threads (forks.pm)
by BrowserUk (Patriarch) on Sep 22, 2010 at 15:42 UTC

otherwise just one page is copied (and afterwards, the copy is modified). As such, this example of events is misleading.

The problem with this view is it kind of implies that (say), iterating a small array will only entail copying one 4k block. The reality is that the AV is likely to be allocated in a different block to the xpvav; which is quite likely to be in a different block to the SV with # magic; which is in a different page to the SV*array containing the alloc; which is in a different place to the block containing all the SVs holding the scalars; which may each be in a different block to the xvpv, xviv or xvnv they point to; which in turn are quite likely to be in different pages to actual pv data if it has any; etc.

And that was just an array. Take a close look at a hash; or better still a stash, and see all the separate allocations that go to making up the thing. And remember, perl's allocator tend to allocate like sized things from the same pools, so the individual elements of any single entity tend to be strewn around over several pages; not all together in one neat lump (page).

So whilst it isn't (nor was I implying), a 1 for 1, 4k block copied for every internal change. Nor do changes to a single entity necessarily only involve the copying of a single block. push, shift, pop or unshift an element to a small array, and several blocks might need to be copied as a result.

The CPU/MMU operations required to throw a page fault and allocate memory are insignificant compared to the time it takes to actually copy the data in memory.

I'm sorry, but I disagree. A page fault is a hugely expensive operation, because of the ring3-ring0-ring3 transition. Copying (within the auspices of a single process) a 4k block or even a 64k block is less expensive.

For an example of just how expensive page faults can be, see What could cause excessive page faults?.

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP an inspiration; A true Folk's Guy

[reply]


Pathologically Eclectic Rubbish Lister
	PerlMonks