http://qs321.pair.com?node_id=651793


in reply to Re^2: Taking advantage of dual processors
in thread Taking advantage of dual processors [tag://parallel,cygwin,multi-core,fork,i/o]

Why don't you try it? make these extra processes happen in a controlled fashion. Even without having a pool of permanent sqlite writers getting input from a queue which would be the right way to do it (as BrowserUK pointed out even though its example uses threads), you can still quickly prototype with the code you've shown using the following: keep an array of n max. commits-to-do and every m <n lines fire m processes system("my_writer $update &"), throttle a bit if ps (or even /proc) gives you too many my_writer processes, increase if your process count drops below a threshold (can be made adaptive...). Decide what to do if you reach n. (slow down, wait for #proc < n_min etc...) cheers --stephan

  • Comment on Re^3: Taking advantage of dual processors