Re: Parsing .2bit DNA files

One thing that strikes me as odd/inefficient is that you are explicily converting to strings of ASCII '0' and '1' to then convert to A, C, G, T. It seems like it would be more direct to convert a byte (4 DNA bases) at a time. Here is a cute way to do that, at the expense of having a lookup table for all 256 values:

my @CONV = glob( "{T,C,A,G}" x 4 );
my $dna = join "", @CONV[ unpack "C*", $raw ];
[download]

On my system, this gives the same output as yours. I don't know if it's better, but it is shorter, and it can conveniently use an array instead of a hash. You could also experiment with different tradeoffs on lookup table sizes:

## takes 16 bits (= 8 bases = unsigned short) at a time
my @CONV = glob( "{T,C,A,G}" x 8 );
my $dna = join "", @CONV[ unpack "S*", $raw ];
[download]

For some reason, I had byte-order issues doing this. Of course, you must also be careful that $raw is padded to a multiple of 16 bits!

Another cute trick I can think of is that you can do some bit-twiddling to implement the M-blocks (apparently lowercasing a range of characters). In ASCII, you can toggle the case of an alphabetic character by bitwise-XOR'ing it with the space character. So I think you can rewrite:

substr($dna, $_, $mblock{$_}, lc(substr($dna, $_, $mblock{$_})))
[download]

substr($dna, $_, $mblock{$_}) ^= (" " x $mblock{$_});
[download]

Alternatively, you could use %mblock to generate a long mask of chr(0)'s and chr(32)'s that you can XOR with the entire $dna. Again, probably not a big deal but certainly higher cute-value.

Of course you could always fix M,N blocks on-the-fly, as you are unpacking them from $raw, but that would require some more work. Since I'm typing one-handed these days and it takes me forever, I think I will pass on playing with some code that does that! ;)

blokhead

Comment on Re: Parsing .2bit DNA files Select or Download Code

Replies are listed 'Best First'.

Re^2: Parsing .2bit DNA files
by bart (Canon) on Mar 06, 2008 at 11:48 UTC

glob

A reliable way to do it would be to generate a list of integers, in this case from 0 to 255, and convert each to a string using base 4 — admittedly, I don't know how to best do it in Perl... As a second step, I'd convert the digits '0' .. '3' to the letters, for example with

tr/0123/TCAG/
[download]

Anyway, you say

at the expense of having a lookup table for all 256 values

You could also experiment with different trade-offs on lookup table sizes:

For some reason, I had byte-order issues doing this.

unpack

I have some doubts about using unpack "C*", $raw to convert the byte sequence into numbers. Ouch. That sequence can be millions or even billions of bytes long, and that is a very long list. I think it's better to convert the $raw string either in short sequences of, say, a few k each (the compromise is in loop count vs. memory usage per loop),

my $dna = '';
use constant CHUNKSIZE => 2048;
for (my $offset = 0; $offset < length($raw); $offset += CHUNKSIZE) {
   $dna.= join '', @CONV[ unpack 'C*', substr $raw, $offset, CHUNKSIZE
+ ];
}
[download]

s///

s/(.)/$CONV[ord $1]/sge
[download]

Finally: don't forget to cut off the junk at the end of the sequence, making the length the same as the number of entries there were expected according to the record header.

[reply]
[d/l]
[select]


more useful options
	PerlMonks