"If I understand you correctly, you have a list of 500 data sets, where each set comprises 1 to 7 matrices of 1K x 1K values (bytes? ints?); your test run of 13 "observation numbers" yielded 51 query executions, so maybe there was an average of about 3 matrices per observation? (13 queries to get the lists of detectors per observation, plus 13 * 3 queries to get the matrices for three detectors, would come to 52 queries -- am I on the right track here?)"
Sortakinda. The main diff is that each chip/matrix/CCD might only have a few 'hits' on it. So what my two DB queries do is:
- for each observation number in the list, return the chips/CCDs that actually have hits;
- for each chip/CCD involved in that observation number, return the x and y value of each hit on the chip/CCD.
Hope that makes more sense. The crunching that follows iterates over the result sets ... but is decoupled from the DB calls.
