Just how many files are reached through this cascade of includes? What's the upper bound — how large is the population of .rules files at your site? And just how fast do you need it to be? I'm currently using an edited version of Module::Dependencies to track the dependencies of a file that, in the end, calls (using include, use, do and backticks) about 55 other files and it takes less than 10 sec on a pentium, and that's while drawing a PostScript graph of all the relationships. Threads sound like overkill to me. If you're having to keep the entire contents of each file in memory and your files are huge (e.g. some combination of Perl code and DATA) then maybe you're getting into some problems, but your recursive approach sounds right to me.

Hi

Thanx for ur replies and coming to the topic actually i am working this out in LINUX and here we are using it temporarily for a file which has around 100 include statements and again in each inlcude file there may be around 1 to 10 ranging depinding on the type of device rules file. So here its just parent and a Child level ......

Firstly we want to test it our for at leat 2 to 3 levels of includes within include files. Coming to the fastness as you said it not a big problem about it will be OK if it takes adequate amount of time.

i shall also jus give out a sample code which is presently backing up the files which are not having any inlcude statements and also recursively searching for the inlcude into the Depth........

#!/usr/bin/perl 

use strict;


   my $first_path = "/opt/netcool/etc/test.rules";
   &srch_include($first_path);
   my ($incl_full_path , $dir_path, $file_name) ;
 

   sub srch_include
     {
        my ($main_path) = @_  ;
        print " First Sub Path : $main_path \n";

        open( FH1, "$main_path") or die " Sub1 Error: $! \n ";
        my @main_file_cont = <FH1>;
        my $count = 0;
         
        foreach my $main_file_line ( @main_file_cont )
          { 
             if( $main_file_line =~ /^(\s+)?include\s+"(.*)"$/)
               {  

                  my $count++;
                  $incl_full_path = $2 ;
                  print " Inlcudes :: $incl_full_path \n ";
     
                  $incl_full_path =~ /(.*\/)(.*)/;
                  $dir_path = $1 ;
                  $file_name = $2;

                  print " Dir : $dir_path  File : $file_name \n ";
                  system ("mkdir -p /root/msk/$dir_path");
                  sleep 2 ;
                  open(CR, ">/root/msk/$incl_full_path") or die " Main
+ File Error: $! \n";
                  print CR " @main_file_cont ";
                  close(CR);

                  &srch_include($incl_full_path);   
                }           
          }

        if($count == 0 )
        { 
             #print " $incl_full_path has No includes and lookup \n ";

             $incl_full_path =~ /(.*\/)(.*)/;
             $dir_path = $1 ;
             $file_name = $2;

             #print " Dir : $dir_path  File : $file_name \n " 
             system ("mkdir -p /root/msk/$dir_path");
             sleep 2 ;
             open(CR, ">/root/msk/$incl_full_path") or die " No Includ
+e Error: $! \n";
             print CR " @main_file_cont ";
             close(CR);
        }
     }
[download]

here i am able to back the files with no inlcude statements and also the files which has inlcudes i am backing up the main file too and then searching and gooing into the depth . So jus check ot out and i shall be waiting for ur replies and thank you very much for ur help....
Work Hard Party Harderrr!!
Sushil Kumar

Are these rules files generated by a program, or have they been hand coded? If the latter, you probably want to check for single quotes around the filename too, rather than just double quotes.

You've got my $count twice (as strict will warn you.) The value of $count isn't going to make it out of the if statement, so your $count == 0 test will always succeed.

In your

$main_file_line =~ /^(\s+)?include\s+"(.*)"$/
[download]

the (\s+)? is obfusciated; \s* is much clearer and efficient.

Are you sure you don't have any circular dependencies in your include statements? You aren't testing for them. If there is one... well, endless loops take a long time to finish. Also, how do you want to handle the case where there are overlapping dependencies? E.g, A includes B and C; B and C each include D. Do you really want D and its dependencies to be searched twice?

throop

Looking at your code, one big problem stands out with regard to recursion. "FH1" needs to be localized in some way (as you'll see in merlyn's article). I think most people today would do this by putting the file handle in a scaler variable. Just put a

my $fh1;
[download]

Another approach (?) is to generate commands for a simple (?) machine. Depending on the size of the things you want to process, the "machine" can be stopped in one place and started again. If the files don't change, you keep a cache of the commands.

For example suppose you just want to read all lines of included files in chunks. One obvious command could be 'read file byte-range'. The command generator would start reading a given file and emit a read command when finding the '^include ' line, push a seek command on a internal stack to come back to the file after the include, and start with the included file right away looking for more includes. When done it would process the stack.

"grep -n '^include '" comes to mind for quick prototyping.