http://qs321.pair.com?node_id=1208000


in reply to Re^3: Memory utilization and hashes
in thread Memory utilization and hashes

The OP did update his data here.

A nit, but I think he wanted the anon array 'Answers' to be an array of hash refs.

He was concerned about creating a hash too large for memory. :-)
If the 'ids' were only 1 to 4, this wouldn't be a problem. But if the different 'ids' were in the thousands or so, I think that was what he what he wanted to avoid. You do reuse hash keys for this small example.

There was only 1 repeated id in his data, and this is where it is reused.

The test for 'exists' therefore tests true for this id only and output gets called. The rest of the output gets done outside the while loop (#remaining).

Your code outputs (on his revised data with my ($s1,$n,$s2,$v2,undef) = split ';',$_,5; changed to my (undef,$s1,$n,$s2,$v2) = split ';',$_,5;):

{"ip":["2.3.4.5","2.3.4.5"],"id":"2","host":"www.cnn.com"} {"ip":["3.4.5.6","1.2.4.5"],"id":"4","host":"www.google.com"} {"ip":["1.2.3.4"],"id":"1","host":"www.example.com"} {"ip":["3.4.5.6"],"id":"3","host":"www.google.com"} {"ip":["2.3.4.5"],"id":"2","host":"www.example2.com"}
I coded a similar solution to you (only difference being an array of hashrefs instead of values alone).
{"id":"1","Answer":[{"ip":"1.2.3.4"}],"host":"www.example.com"} {"id":"2","Answer":[{"ip":"2.3.4.5"},{"ip":"2.3.4.5"}],"host":"www.cnn +.com"} {"id":"3","Answer":[{"ip":"3.4.5.6"}],"host":"www.google.com"} {"id":"4","Answer":[{"ip":"3.4.5.6"},{"ip":"1.2.4.5"}],"host":"www.goo +gle.com"} {"id":"2","Answer":[{"ip":"2.3.4.5"}],"host":"www.example2.com"}
My solution was:
#!/usr/bin/perl use strict; use warnings; use JSON; open my $fh, '<', 'file2' or die $!; my %query; while (<$fh>) { chomp; my (undef, $cat, $id, undef, $val) = split /;/; if ($cat eq 'Query') { if (%query) { output(\%query); %query = (); } $query{$id} = [$val]; } else { push @{ $query{$id} }, {ip => $val}; } } output(\%query); sub output { my $query = shift; my ($id) = keys %$query; my $host = shift @{ $query->{$id} }; print encode_json({Answer => $query->{$id}, id => $id, host => $ho +st}),"\n"; }