delete will definitely reduce the size of the hash, because every time you get a first answer for a given query, it will delete the entire entry for that query. Of course, if there's a second answer for the query, it cannot find the entry for the query, so it creates it again, without the host key.
You might want to expand your example data to include a sample with more than one response (out of order) for the same query (for example, query 2, with two or three rows of answers), and display the output. Then tell us what you want the real output to be, given that set of data. Something like:
Query;1;host;www.example.com
Answer;1;ip;1.2.3.4
Query;2;host;www.cnn.com
Query;3;host;www.google.com
Answer;2;ip;2.3.4.5
Answer;3;ip;3.4.5.6
Answer;2;ip;9.8.7.6
Answer;2;ip;5.4.3.2
-----------------------
{"host":"www.example.com","ip":"1.2.3.4"}
{"ip":"2.3.4.5","host":"www.cnn.com"}
{"ip":"3.4.5.6","host":"www.google.com"}
{"ip":"9.8.7.6"}
{"ip":"5.4.3.2"}
Also, for debugging, add print "DEBUG: ", encode_json \%pairs; just before the end of the while loop: that will let you watch the hash grow and shrink, and will tell you whether or not it's doing the right thing