Beefy Boxes and Bandwidth Generously Provided by pair Networks
Problems? Is your data what you think it is?
 
PerlMonks  

Should I be using tie for this?

by Plankton (Vicar)
on Jun 16, 2004 at 20:35 UTC ( [id://367419]=perlquestion: print w/replies, xml ) Need Help??

Plankton has asked for the wisdom of the Perl Monks concerning the following question:

Friends,

My script uses way too much memory. I haven't ever use tie before. So I thought I could use Tie::File to store my data to disk rather than memory. Here's what I tried to do ... 
#!/usr/local/bin/perl -w
use strict;
use Data::Dumper;
use Tie::File;
use lib('.');

use board;
use hole;
use boardTree;

#
# create the holes
#
my $holeLevel = 0;
my @holes;
for my $i (0..15){
    $holeLevel = 1 if $i >0;
    $holeLevel = 2 if $i >2;
    $holeLevel = 3 if $i >5;
    $holeLevel = 4 if $i >9;
    push ( @holes, new hole( 'white', $i, $holeLevel, undef ) );
}
#
# link the holes
#
$holes[0]->setLinks(  [ $holes[1], $holes[2] ] );
$holes[1]->setLinks(  [ $holes[0], $holes[2], $holes[3], $holes[4] ] )
+;
$holes[2]->setLinks(  [ $holes[0], $holes[1], $holes[4], $holes[5] ] )
+;
$holes[3]->setLinks(  [ $holes[1], $holes[4], $holes[6], $holes[7] ] )
+;
$holes[4]->setLinks(  [ $holes[1], $holes[2], $holes[3], $holes[5], $h
+oles[7], $holes[8] ] );
$holes[5]->setLinks(  [ $holes[2], $holes[4], $holes[8], $holes[9] ] )
+;
$holes[6]->setLinks(  [ $holes[3], $holes[7], $holes[10], $holes[11] ]
+ );
$holes[7]->setLinks(  [ $holes[3], $holes[4], $holes[6], $holes[8], $h
+oles[11], $holes[12] ] );
$holes[8]->setLinks(  [ $holes[4], $holes[5], $holes[7], $holes[9], $h
+oles[12], $holes[13] ] );
$holes[9]->setLinks(  [ $holes[5], $holes[8], $holes[13], $holes[14] ]
+ );
$holes[10]->setLinks( [ $holes[6], $holes[11] ] );
$holes[11]->setLinks( [ $holes[6], $holes[7], $holes[10], $holes[12] ]
+ );
$holes[12]->setLinks( [ $holes[7], $holes[8], $holes[11], $holes[13] ]
+ );
$holes[13]->setLinks( [ $holes[8], $holes[9], $holes[12], $holes[14] ]
+ );
$holes[14]->setLinks( [ $holes[9], $holes[13] ] );

my $BT = new boardTree ( new board( \@holes, 0 ), 1, 1, [ new boardTre
+e( new board ( undef, 1 ), 1, 1, [] ) ] );

#
# lets try tie the boardTree object to a file
#
tie $BT, 'Tie::File', "thetiefile";

# I am guess that the script is dying because
# this will use up too much virtual memory and 
# the OS (Window XP) kills the script.
$BT->{'links'}[0]->makeLinks();
# I never get here because of memory issue
$BT->dumpPretty();

[download]
... here's the output I get ... 
$ ./tieBoardTree.pl
Can't locate object method "TIESCALAR" via package "Tie::File" at
        ./tieBoardTree.pl line 47 (#1)
    (F) You called a method correctly, and it correctly indicated a pa
+ckage
    functioning as a class, but that package doesn't define that parti
+cular
    method, nor does any of its base classes.  See perlobj.
    
Uncaught exception from user code:
        Can't locate object method "TIESCALAR" via package "Tie::File"
+ at ./tieBoardTree.pl line 47.

[download]
Like I said I have never used tie before. Maybe tie is completely the wrong thing to use here. If so what should I do?

Plankton: 1% Evil, 99% Hot Gas.

edit (broquaint): changed <pre> tags to <code>

Replies are listed 'Best First'.
Re: Should I be using tie for this?
by Aragorn (Curate) on Jun 16, 2004 at 20:57 UTC
    Maybe tie is completely the wrong thing to use here. If so what should I do?
    It is indeed the wrong thing to do. What Tie::File does is provide an array interface to a file on disk. You can use huge files as an array because Tie::File only holds a part of the file in memory, swapping other parts in and out as necessary.

    In your code I see the comment 

    # I am guess that the script is dying because
# this will use up too much virtual memory and 
# the OS (Window XP) kills the script.

    [download]
    Your first task is to find out why it is taking up so much space. Use the debugger to step through you program and see where it fails.

    Just assuming that something is wrong somewhere without determining the exact cause but throwing some code at the problem is not going to fix your program.

    Arjen

[reply]
[d/l]
      My memory problem comes from the nature of what my script is trying to do. I am trying to map out all possible moves in Tam's Chinese Peg Game. I didn't think I would have a memory problem. I guess I should of stayed awake in combinatorics. Anyways if you got any suggestion I'd sure appreciate it.

      Plankton: 1% Evil, 99% Hot Gas.
[reply]
Re: Should I be using tie for this?
by thor (Priest) on Jun 16, 2004 at 20:51 UTC
    Tie::File allows one to access a file as thought it were an array; one line of the file per array element. I don't know what a boardTree object is, but if you have control over over its source, you can have it write things to a file and access them from there via a tied interface...but only if it makes sense to access them as an array (which it looks like it might based on how you are initializing your array). Come to think of it, couldn't you tie your @holes array to Tie::File?

    thor

[reply]
      Thanks thor,
      That was a good suggestion. Unfortunately it didn't work. When my script executes ... 
      #
# link the holes
#
$holes[0]->setLinks(  [ $holes[1], $holes[2] ] );

      [download]
      ... it generates this error message ... 
      Can't locate object method "setLinks" via package "hole=HASH(0x1a1c2e4
+)
" (perhaps you forgot to load "hole=HASH(0x1a1c2e4)
"?) at ./tieBoardTree.pl line 26, <$fh> line 3.

      [download]

      Plankton: 1% Evil, 99% Hot Gas.
[reply]
[d/l]
[select]
Re: Should I be using tie for this?
by Anonymous Monk on Jun 16, 2004 at 20:44 UTC
    perldoc Tie::File 
    # This file documents Tie::File version 0.97
        use Tie::File;

        tie @array, 'Tie::File', filename or die ...;

        $array[13] = 'blah';     # line 13 of the file is now 'blah'
        print $array[42];        # display line 42 of the file

    [download]
    ARRAY ARRAY ARRAY ARRAY
[reply]
[d/l]
      Thanks. I saw that. I guess I should restate my question. Can I use Tie::File with a object instead of an array. And assuming that ...

      ARRAY ARRAY ARRAY ARRAY

      ... is not Anony Monks's sig. I guess you are saying I cannot use Tie::File with anything but an array. Right?

      Plankton: 1% Evil, 99% Hot Gas.
[reply]

        Let's back up to this: "store my data to disk rather than memory". OK, so let's say that you do successfully store your data to disk. Now what? I mean, how do you use it in your program? You have to read it back into memory. Now you have a new problem -- which parts to pull from the file into memory and which parts to move from memory into the file. Tie::Cache::LRU might make this task easier, however.

        Sounds to me like you are trying to do too much. Try filtering the results down, or use some sort of pagination scheme to only deal with a small window of the entire set, instead of the entire set. As far as storing "objects" ... look into Serialization: Freeze::Thaw and YAML come to mind.

        jeffa

        L-LL-L--L-LL-L--L-LL-L--
-R--R-RR-R--R-RR-R--R-RR
B--B--B--B--B--B--B--B--
H---H---H---H---H---H---
(the triplet paradiddle with high-hat)
[reply]

Back to Seekers of Perl Wisdom

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?

www.com | www.net | www.org
Node Status?
node history
Node Type: perlquestion [id://367419]
Approved by Aragorn
help
Chatterbox?
and the web crawler heard nothing...

How do I use this? • Last hour • Other CB clients
Other Users?
Others admiring the Monastery: (7)
As of 2024-04-24 09:50 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found