Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

perl database question

by baxy77bax (Deacon)
on Apr 26, 2008 at 17:59 UTC ( [id://683070]=perlquestion: print w/replies, xml ) Need Help??

baxy77bax has asked for the wisdom of the Perl Monks concerning the following question:

hi,

wel my question is : does perl has something like "Gadfly" -> it allows Python programs a convenient way to store, retrieve and query tabular data without having to rely on any external database engine or package.

or maybe something similar to it and that isn't in-memory relational database style engine like "Gadfly"

robert

Replies are listed 'Best First'.
Re: perl database question
by kyle (Abbot) on Apr 26, 2008 at 19:12 UTC

    That sounds like DBD::SQLite, which will also require DBI. Both are very common.

      thanks,

      i'm asking this because i don't need for 10 people to log on to my database or todo some complex data mining. i need a speed of a mysql data retrieval and to be able to deal with very very large data sets. also i would be the only user and there would be only one database so also i don't see the need for connecting protocol.

      if there is something it that domain for perl programming it would be very helpfull

      r

        See, again, DBD::SQLite, and maybe http://sqlite.org. Also, consider maybe DBM::Deep or one of the other hash solutions. So far, you haven't told us what your needs are, and I'm too lazy to dust off my crystal ball just for you.

Re: perl database question
by Your Mother (Archbishop) on Apr 26, 2008 at 19:37 UTC

    Depending on what you're doing, plain old Storable + apl's suggestion of a hash of arrays or something will be very zippy and easy. There are other non-SQL DB solutions like DB_File (needs Berkeley DB underneath) or, probably much better, DBM::Deep. Like kyle said, SQLite is the way to go if you need SQL. I'd say unless you plan to transition the app/script to a real database eventually, you probably don't (unless you're much better with SQL than with Perl data structures, in which case it would save you dev time).

Re: perl database question
by Cody Pendant (Prior) on Apr 26, 2008 at 21:31 UTC
    I'm not sure why you say Gadfly allows python users to work on data without installing an external package. Gadfly is an external package, surely?

    Anyway, what about DBD::AnyData's in-memory table option?



    Nobody says perl looks like line-noise any more
    kids today don't know what line-noise IS ...
Re: perl database question
by ww (Archbishop) on Apr 26, 2008 at 20:05 UTC

    Your question would have been much clearer had it included a description of Gadfly rather than making us guess, "dust off the crystal ball" or Google for an explanation:

    GadflyB5: SQL Relational Database in Python
    Gadfly is a simple relational database system implemented in Python based on the SQL Structured Query Language.

    That said, it's still not clear if you really mean SQL or any kind of RDBMS or whether you want to work with TSV or CSV data sources.

    If the other answers in this thread aren't going the way you want, you might search CPAN for modules in those families.

      ok

      i see that crystal balls had been dusting them selves. the cvs , tvs, -> these type of files are NO GO. SOL would be nice but RDBMS still comes into consideration. i thought i was clear enough when i defined gadfly, but now it looks like i wasn't, sorry -> nasty habit. so as i said i need something that is like perl module (meaning inside of perl - that doesn't require separate installation procedure and configuration like mysql -- for example), and that acts as database engine. meaning that i can query large tables(several gb), connect data from different tables and insert data into new tables. i would have only one db, and only my application would move around it. so as i said i need db engine like mysql but that can contain only one database, several tables, can do searching through tables, does not have that silly 'connect to the database' procedure, it is fast(at least as mysql) and deals well with very very large data.

      am i asking too much ?

      SQLite looks promising but i still have to read the whole documentation

      anyway thanks

      r

        ...does not have that silly 'connect to the database' procedure,...

        I'll second (or third or fourth) the motion for (DBI and) DBD::SQLite. Yes, you have to install the module, but it's not the sort of big database engine comparable to Oracle, Sybase, or even MySQL. And you do have to go through the motions of connecting to the database, but that is because it's accessible through the perl DBI module (after all, you have to tell DBD::SQLite where your database file is somehow), and DBI is the standard way to 'connect' to almost any SQL database. It should be an easy module to install, and IMHO SQLite is the reason that perl doesn't need anything like Gadfly (or at least there is very little need). There was some activity to make a pure perl database sometime ago, but it kind of died out, and SQLite fulfills most of the need for that sort of module.

Re: perl database question
by apl (Monsignor) on Apr 26, 2008 at 19:27 UTC
      how do hashes deal with files up to several GB in size, i didn't have pleasant experience with hashes (too slow and too RAM consuming)
        how do hashes deal with files up to several GB in size
        You will note this is the first time you mentioned gigs of data. What you had asked for was
        a convenient way to store, retrieve and query tabular data without having to rely on any external database engine or package.
        Hashes satisfy all four requirements quite nicely.

        Had you included this new requirement, I would have gone with one of the CPAN modules, even though that violates your fourth requirement.

Re: perl database question
by shmem (Chancellor) on Apr 27, 2008 at 12:14 UTC

    Have a look at DBM::Deep.

    --shmem

    _($_=" "x(1<<5)."?\n".q·/)Oo.  G°\        /
                                  /\_¯/(q    /
    ----------------------------  \__(m.====·.(_("always off the crowd"))."·
    ");sub _{s./.($e="'Itrs `mnsgdq Gdbj O`qkdq")=~y/"-y/#-z/;$e.e && print}

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://683070]
Approved by stiller
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (7)
As of 2024-03-28 10:44 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found