Retriving BerkeleyDB Data

hawk2379 has asked for the wisdom of the Perl Monks concerning the following question:

Hello I am trying to read data from a BerkeleyDB file assocciated with a program called QTstalker. With the code below, I wanted to print the key and value with the use of perl. My keys come out fine but my values are gibberish.

#!/usr/bin/perl
use strict;
use BerkeleyDB;

my $filename = '/home/user/Perl/AA';
my %h ;
tie %h, 'BerkeleyDB::Btree',
          -Filename   => $filename,
          -Flags      => DB_CREATE
or die "Cannot open $filename $! $BerkeleyDB::Error" ;

foreach $key (keys %h)
{
    print "$key:";
    print "$h{$key}\n";
}
[download]

Sample output: "20140528000000:�Q��*@+@�G�z�*@= ףp�*@Z�aA 20140529000000:��(\*@)\�(+@�p= ף*@)\�(+@o�\A 20140904000000:)\�(*@R��Q*@)\��)@*@��ʇA"

Some info I gathered from QTstalker's SourceForge forums: "The btree structure is simple enough: Each chart has it's own file somewhere in .qtstalker/data1/data, each entry in the file is a single bar of data: The keys are strings in this format: "yyyymmddHHMMSS" (eg. "20080128013000", self-explanatory) The value behind each key are 48 bytes where the first 40 bytes are five doubles representing (open,high,low,close,volume) of the bar. Last 8 bytes is a long integer representing open interest."

Ultimately, I want read the key/value pairs in human form(possibly export to an excel file) and also insert key/values that the database can store for QTstalker. If someone is kind enough to explain what I am doing wrong and decode what Im looking at, I would greatly appreciate it.

Comment on Retriving BerkeleyDB Data Download Code

Replies are listed 'Best First'.
Re: Retriving BerkeleyDB Data by roboticus (Chancellor) on Aug 02, 2014 at 01:21 UTC
hawk2379: It sounds like you'll just need to use the unpack function (`perldoc -f unpack`) to pull the byte string apart into the fields you want. I'd guess it would be something like: `my ($datetime, $open, $high, $low, $close, $volume, $interest) = unpack "A14f5Q", $h{$key};` [download] Note: Totally untested, but I think that if you play around with unpack you'll be able to pull it apart. (You'll want to check `perldoc -f pack` and `perldoc perlpacktut` as well.) Update: changed variable names. ...roboticus When your only tool is a hammer, all problems look like your thumb.	[reply] [d/l] [select]
Re^2: Retriving BerkeleyDB Data by hawk2379 (Novice) on Aug 02, 2014 at 02:40 UTC
I was able to write up a rough script based on your recommendations. And it works! `#!/usr/bin/perl use BerkeleyDB; print "start\n"; my $filename = '/home/user/Perl/AA'; #unlink $filename ; my %h ; my $dbh = tie %h, 'BerkeleyDB::Btree', -Filename => $filename, -Flags => DB_CREATE or die "Cannot open $filename $! $BerkeleyDB::Error" ; foreach $key (keys %h) { my ($open, $high, $low, $close, $volume, $interest) = unpack "dddddl", $h{$key}; print "$key: $open,$high,$low,$close,$volume,$interest\n"; # print "$h{$key}\n"; } untie %h; print "end\n";` [download] Thank You for your help.	[reply] [d/l]


"be consistent"
	PerlMonks