Unification of Directories (their contents)

PetaMem has asked for the wisdom of the Perl Monks concerning the following question:

Hi,

given n directories D1..Dn with arbitrary content. I would like a function that takes these source directories and copies their contents to a destination directory Dd such, that Dd is the union of all files and the fs structure of D1..Dn.

D1
 - Folder1
   - file1
   - file2
 - Folder2
   - file4

D2
 - Folder1
   - file3
   - file5
 - Folder3
   - file7

=>

Dd
 - Folder1
   - file1
   - file2
   - file3
   - file5
 - Folder2
   - file4
 - Folder3
   - file7

Right now, I can assume, that the sets of D1..Dn are mutually exclusive, but of course a hook on how to behave on clashes (and what type of clash .. name/diff) would not hurt.

Does such a thing already exist? I know about UnionFS, but rather than this transparent solution I would prefer a CLI-based batch processing tool, because these merging operations are one-shot only.

Update:

Maybe I should start to have LESS Perl on my mind... ;-)

> cp -a D[12]/* Dd/
[download]

Does the trick. Ok, it's not that portable, it's not perl, but for the moment...

Bye
PetaMem All Perl: MT, NLP, NLU

Comment on Unification of Directories (their contents) Download Code

Replies are listed 'Best First'.
Re: Unification of Directories (their contents) by graff (Chancellor) on Apr 26, 2009 at 04:09 UTC
If I understand it correctly, "cp -a" is linux/gnu only -- bsd (including freebsd and macosx) don't have a "-a" option for "cp" (though other options in combination will give the same result). Note that your usage (`cp -a D[12]/* Dd/`) will take the "depth 1" contents in ascii-betic order, and deeper contents in "arbitrary" (directory content) order, so in any case, file age doesn't enter into the logic. If there might be path/name collisions among the source directories, where differences of age and content matter, you might look at "rsync" (for which you'll find perl modules on CPAN, in case that helps, but you'll probably need to grok the rsync command-line tool options anyway, so try "man rsync").	[reply] [d/l]
Re: Unification of Directories (their contents) by atcroft (Abbot) on Apr 26, 2009 at 09:57 UTC
My thoughts upon reading this (which I hope would prove helpful) were : Use File::Find to generate a list of files (also potentially at this point exiting or asking for processing options for duplicate file names) Use the -d dirname test, mkdir() and File::Spec to create subdirectories that may be needed Use File::Copy's copy() function for the final movement At least, that's the process I would likely procede with (assuming a better solution were not proposed). HTH.	[reply]
Re: Unification of Directories (their contents) by spx2 (Deacon) on Aug 07, 2009 at 10:17 UTC
There is a solution for your problem , however some corners have not been detailed enough. For example , what if there are 2 files , one in D1 and one in D2 , and they don't sit in the same place ( relative to D1 and D2 ) but they have the same content . Would that be considered a duplicate or not ? If the answer to that question is no then you can use shlomif's nice method for recursive diff of 2 directories found here and tailor that to your needs. Otherwise , it may be more difficult and you might need to calculate some hashes for all files(in both D1 and D2) and start seeing if the files have been included before in Dd or not.	[reply]


Come for the quick hacks, stay for the epiphanies.
	PerlMonks