Beefy Boxes and Bandwidth Generously Provided by pair Networks
Perl-Sensitive Sunglasses
 
PerlMonks  

Re: Get random unique lines from file

by Marshall (Canon)
on Aug 17, 2012 at 22:06 UTC ( [id://988105]=note: print w/replies, xml ) Need Help??


in reply to Get random unique lines from file

The formatting of the code/data and intent was hard for me to understand. But maybe this works for you?:
#!/usr/bin/perl -w use strict; use Data::Dumper; # a core utility use List::Util qw(shuffle); # a core utility my @data; while (<DATA>) { next if /^\s*$/; #skip blank lines my ($important) = (split(' ',$_))[0]; push @data, $important; } print Dumper \@data; #debug - just to verify the split() # so @data is about 100,000 entries # this is a big array, but not that big # the use of splice() works, but is relatively inefficient # perhaps just use List::Util shuffle to randomize the # indicies? # This will create a list with 100,000 values # but it will do it efficiently. # Within the parameters of what you have said, # 1/4 million array/list elements will work ok, even on # my wimpy Win XP32 bit machine foreach my $index (shuffle (0..@data-1)) { print "$index $data[$index]\n"; } =prints VAR1 = [ 'FRZPY5Q02F00L9.ICM_AWP_Bv6.AWP_0001_2007_08_23', 'FRZPY5Q02F00U8.ICM_AWP_Bv6.AWP_0001_2007_08_23', 'FRZPY5Q02F01NC.ICM_AWP_Bv6.AWP_0001_2007_08_23', 'FRZPY5Q02F023C.ICM_AWP_Bv6.AWP_0001_2007_08_23' ]; 3 FRZPY5Q02F023C.ICM_AWP_Bv6.AWP_0001_2007_08_23 2 FRZPY5Q02F01NC.ICM_AWP_Bv6.AWP_0001_2007_08_23 1 FRZPY5Q02F00U8.ICM_AWP_Bv6.AWP_0001_2007_08_23 0 FRZPY5Q02F00L9.ICM_AWP_Bv6.AWP_0001_2007_08_23 =cut __DATA__ FRZPY5Q02F00L9.ICM_AWP_Bv6.AWP_0001_2007_08_23 ACTGCCAACGCGCAGAACCTTAC +CAGGTCCTGACTTCCTGACTATGGTTATTAGAAATAA TTTCCTTCAGTTCGGCTGGGTCAGTGACAGG +TGATGCATGGCCGTC FRZPY5Q02F00U8.ICM_AWP_Bv6.AWP_0001_2007_08_23 ACTGCCTAACCGATGAACCTTAC +CTACACTTGACATGCAGAGAACTTTCCAGAGATGGAT TGGTGCCTTCGGGAACTCTGACACAGGTGAT +GCATCGCCGTC FRZPY5Q02F01NC.ICM_AWP_Bv6.AWP_0001_2007_08_23 ACTGCCTACGCGAAGAACCTTAC +CTACACTTGACATACAGAGAACTTACCAGAGATGGTT TGGTGCCTTCGGGAACTCTGATACAGGTGAT +GCATGGCTGTC FRZPY5Q02F023C.ICM_AWP_Bv6.AWP_0001_2007_08_23 ACTGCCAACGCGCAGAACCTTAC +CAACCCTTGACATCCAGAGAATTTTCTAGAGATAGAT TTGTGCCTTCGGGAACTCTGTGACAGGTGAT +GCATGGCTGTC

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://988105]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others taking refuge in the Monastery: (3)
As of 2024-04-19 22:26 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found