Beefy Boxes and Bandwidth Generously Provided by pair Networks
Clear questions and runnable code
get the best and fastest answer

Retriving BerkeleyDB Data

by hawk2379 (Novice)
on Aug 01, 2014 at 23:58 UTC ( #1095984=perlquestion: print w/replies, xml ) Need Help??

hawk2379 has asked for the wisdom of the Perl Monks concerning the following question:

Hello I am trying to read data from a BerkeleyDB file assocciated with a program called QTstalker. With the code below, I wanted to print the key and value with the use of perl. My keys come out fine but my values are gibberish.
#!/usr/bin/perl use strict; use BerkeleyDB; my $filename = '/home/user/Perl/AA'; my %h ; tie %h, 'BerkeleyDB::Btree', -Filename => $filename, -Flags => DB_CREATE or die "Cannot open $filename $! $BerkeleyDB::Error" ; foreach $key (keys %h) { print "$key:"; print "$h{$key}\n"; }

Sample output: "20140528000000:�Q���*@+@�G�z�*@= ףp�*@Z�aA 20140529000000:��(\*@)\�(+@�p= ף*@)\�(+@o�\A 20140904000000:)\�(*@R���Q*@)\��)@*@��ʇA"

Some info I gathered from QTstalker's SourceForge forums: "The btree structure is simple enough: Each chart has it's own file somewhere in .qtstalker/data1/data, each entry in the file is a single bar of data: The keys are strings in this format: "yyyymmddHHMMSS" (eg. "20080128013000", self-explanatory) The value behind each key are 48 bytes where the first 40 bytes are five doubles representing (open,high,low,close,volume) of the bar. Last 8 bytes is a long integer representing open interest."

Ultimately, I want read the key/value pairs in human form(possibly export to an excel file) and also insert key/values that the database can store for QTstalker. If someone is kind enough to explain what I am doing wrong and decode what Im looking at, I would greatly appreciate it.

Replies are listed 'Best First'.
Re: Retriving BerkeleyDB Data
by roboticus (Chancellor) on Aug 02, 2014 at 01:21 UTC


    It sounds like you'll just need to use the unpack function (perldoc -f unpack) to pull the byte string apart into the fields you want. I'd guess it would be something like:

    my ($datetime, $open, $high, $low, $close, $volume, $interest) = unpack "A14f5Q", $h{$key};

    Note: Totally untested, but I think that if you play around with unpack you'll be able to pull it apart. (You'll want to check perldoc -f pack and perldoc perlpacktut as well.)

    Update: changed variable names.


    When your only tool is a hammer, all problems look like your thumb.

      I was able to write up a rough script based on your recommendations. And it works!
      #!/usr/bin/perl use BerkeleyDB; print "start\n"; my $filename = '/home/user/Perl/AA'; #unlink $filename ; my %h ; my $dbh = tie %h, 'BerkeleyDB::Btree', -Filename => $filename, -Flags => DB_CREATE or die "Cannot open $filename $! $BerkeleyDB::Error" ; foreach $key (keys %h) { my ($open, $high, $low, $close, $volume, $interest) = unpack "dddddl", $h{$key}; print "$key: $open,$high,$low,$close,$volume,$interest\n"; # print "$h{$key}\n"; } untie %h; print "end\n";
      Thank You for your help.

Log In?

What's my password?
Create A New User
Node Status?
node history
Node Type: perlquestion [id://1095984]
Approved by toolic
and the web crawler heard nothing...

How do I use this? | Other CB clients
Other Users?
Others avoiding work at the Monastery: (2)
As of 2021-01-19 02:42 GMT
Find Nodes?
    Voting Booth?