http://qs321.pair.com?node_id=714860


in reply to Re: What the flock!? Concurrency issues in file writing.
in thread What the flock!? Concurrency issues in file writing.

Not NFS.

I am only sorting to get the data back in line after parallel writing, since data gets written out of order (and is not fixed, so I can't just seek to the proper position and write)

                - Ant
                - Some of my best work - (1 2 3)

  • Comment on Re^2: What the flock!? Concurrency issues in file writing.

Replies are listed 'Best First'.
Re^3: What the flock!? Concurrency issues in file writing.
by BrowserUk (Patriarch) on Oct 01, 2008 at 21:01 UTC

    If you have to sort the results anyway, just dodge the bullet and have each of your processes write to separate files. Then sort the files into one. You can't sort until you get the last line anyway, so there is no loss of parallelism.

    Also, a threaded solution is possible if you are interested.

    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
[reply]
      I've thought about both of these, and I may actually look into the multiple file option since I could possibly write these subfiles in order, if I tweaked everything right.

      Threading is definately something I have thought of but would be a big change, not to mention getting up to speed on threads :)

                      - Ant
                      - Some of my best work - (1 2 3)

[reply]

In Section Seekers of Perl Wisdom