http://qs321.pair.com?node_id=1211160


in reply to Re: Perl Program to efficiently process 500000 small files in a Directory (AIX)
in thread Perl Program to efficiently process 500000 small files in a Directory (AIX)

Would it be faster if you do reading and moving using a shell script?

Something like:

for afile in $dir/*; do achannel=$(awk -F'|' '{print $20}' "${afile}") mv "${afile}" "${outdir}/${achannel}" done

Unlikely. Creating an awk and a mv process for each of half a million files sums up to spawning a million processes. I doubt that this idea will be faster than running a single process (perl script), even on an insanely fast machine.

Alexander

--
Today I will gladly share my knowledge and experience, for there are no sweeter words than "I told you so". ;-)