#!/usr/bin/perl
use strict;
use warnings;

my $CHR_NUM = $ARGV[0];

my $file_location = "/home/ALL.chr$CHR_NUM.phase1_release_v3.20101123.
+snps_indels_svs.genotypes.vcf.gz";

my @id_files;           # you might find a better variable name for th
+at array of arrays
for my $x (1..75) {     # $x is a poor name for a variable, perhaps $f
+ile_nr would be better
        my $file_in = "/home/raj/HAPLOTYPES_JAN_2015/PROG_1_Approach_2
+/C22-$x";
        open my $FH, $file_in or die "Cannot open $file_in $!"; # if o
+ne fails, you might be happy to know which one of the 75 files
        while (my $line = <$FH>){
                chomp $line;
                push @{$id_files[$x]}, $line;
        }
        close $FH;
}
[download]

0  ARRAY(0x600500ae8)
   0  empty slot   # $x starts as 1, so no element for subscript 0
   1  ARRAY(0x600500c68)
      0  'line1 file 1'
      1  'line2 file 1'
      2  ' ... etc.)
   2  ARRAY(0x600500b30)
      0  'line 1 file 2'
      2  ' ... etc.

   ... etc.

   75 ARRAY(0x6005.....)
      0  'line 1 file 75'
      2  ' ... etc.
[download]

print $id_files[$x][2];
[download]

I am not going any further i your code because I don't understand your code, we would really need to see samples of the input data to understand what you do.

Please use meaningful variable names, this will make your life much easier.

One last comment. This "goto"! Besides the fact that using such goto is usually very much frown upon in general and has been frown upon for about 40 to 45 years (I have never felt I needed to use one in Perl, well, at least not this form of goto which tends to break of the tenets of structured programming), it seems to make little sense to set $choice to 0 and, immediately thereafter, to gototo a place where you test whether $choice is equal to 1. This really looks like a design defect. If you just need to exit all the nested loops, there are better ways to control that, using last or sometimes next, in this case probably with a label.

I hope this helps.

Thank You Laurent and Everyone else. All of your suggestions are very helpful and have already improved my abilities. I will get samples of the data input and rewrite my code before i post again. TYVM

Are the IDs in the 75 files unique to each file or does the same ID appear in more than 1 file ?

I assume the genotypes.vcf.gz file is several GB's, is that correct ?

poj

I would suggest a complete overhaul, using strict and warnings, declaring meaningful variable names.

To optimize memory consumpion, read the 1000-line file into memory, then read and process each 50-line file (1000 < 50 * 50).

Please re-post when you have a new outline, and request review when you have something that compiles, but may have a specific problem (which I'm sure you will encounter). This is part of the learning process.

Welcome to The Monastery A1 Transcendence,

Before your program is wheeled into the theater, here is the first wisdom to live by using perl!
Always use these two lines in your Perl Program

use warnings;
use strict;
[download]

Today i am writing in order to obtain some wisdom on my current topic. My task is to write a fairly simple Perl program that 1) Opens a large 1000 genome file. 2) Opens 75 different Files each with 50 IDentifiers 3) In a loop...Starting with File 1 Identifier 1 if (File 1 ID 1 matches 2nd column of the 1000 genome file) extract the whole line from the 1000 genome file and store it in an output that is labeled File1_out. Lastly increase the ID to the 2nd element of 50. Do this for all the identifier in File 1, then after 50th ID open File2 and do the same thing again. Basically do this for all of the 75 Files. I think i should only open each of the files one time and work with arrays.

You have not included samples from these files, that would have significantly helped us visualize your idea. Is the second column in the 1000 genome file a checksum val or ? Or does this file have another custom format ? How about the other 75 files, what do they have?

My suggestion is to utilize hashes since you're not concerned with the ordering of the values in the 1K genome file but rather whether or not these values have matches in the other files. Hashes facilitate quick searching. So for the 1K file, read the second column into a hash:

#Untested code for lack of example input by the OP
use strict; #enforces predeclaration of variables, better scoping.
use warnings; #tells you of errors or violations in your code
use Data::Dumper; #visualize your data structures

my %hash; #declaring a global variable to hold the desired column
open(my $fh, "<", "1k_genome_file.bas") or die("could not open file $!
+\n");
while(my $line = <$fh>){
    chomp $line;
     #split around a delimiter (a space, a tab, a comma...etc).
     my @array=split(/\t/,$line);

     #get the second column
     my $second_column=$array[1];

     $hash{$second_column}=1; 
 }
print Dumper(\%hash) #see if the hash looks like what is expected.
[download]

In the line $hash{$second_column}=1;: entries in second column are used as hash keys, giving each entry a value of 1. Repeated entries will be overwritten, this way you avoid duplicated lines in the 1k file. Alternatively, if your second column is made up of unique entries, you can use the line number where the entry in the file occurred as your hash value (that will be helpful later when you want to access the lines to be extracted). Here is a list of relevant posts reading the file using the line number, Line number in a file and Best way to read line x from a file.

Now that you have read the desired column into a hash, you can then iterate over the other files in the folder and for each file check if the hash keys match that line then extract them from the 1k file. The module File::Find is a friendly way to iterating over folders.

As other monks have suggested, turning on strict and warnings is a good coding behavior

Lines 7 through 11: interesting use of a symbolic reference to create arrays named @1, @2, and so on. Stopped reading from that point on.

BTW: You didn't mention; does this code actually do what it is supposed to do? It would certainly benefit from a rewrite using modern Perl techniques (test open return values always, use lexical filehandles, use lexical variables, be strict/warnings compliant, don't use symbolic refs, and so on), but if it's already working.........