The stupid question is the question not asked | |
PerlMonks |
Re^5: Rosetta Code: Long List is Long (llil5.ijs vs llil4vec.cpp)by marioroy (Prior) |
on Jan 18, 2023 at 16:47 UTC ( [id://11149672]=note: print w/replies, xml ) | Need Help?? |
What a delight for our Anonymonk friend to come back. Thanks to you, we tried parallel :). ... but files can be unequal sizes, or just one huge single file. I think serious solution would probe inside to find newlines at approx. addresses, then pass chunks coords to workers to parse in parallel. Chuma mentions 2,064 input files in the initial "Long list is long" thread. Processing a list of files in parallel is suited for this use case due to many files. Back in 2014, I wrote utilities that support both chunking and list modes; mce_grep and egrep.pl via --chunk-level={auto|file|list}. llil5p.ijs I took llil5.ijs and created a parallel version named llil5p.ijs, based on code-bits from your prior post. The number of threads can be specified via the NUM_THREADS environment variable.
llil5tp.ijs Next, I applied the turbo update to the parallel version and named it llil5tp.ijs.
In Section
Meditations
|
|