Beefy Boxes and Bandwidth Generously Provided by pair Networks
Don't ask to ask, just ask
 
PerlMonks  

Where to put an application database?

by boftx (Deacon)
on Oct 16, 2014 at 04:43 UTC ( [id://1104002]=perlquestion: print w/replies, xml ) Need Help??

boftx has asked for the wisdom of the Perl Monks concerning the following question:

Given that you want to have your CPAN application use an SQLite database, what directory would you place the database files in?

One might think that having it under the module directory would be a good choice, but that would almost certainly require root access unless one is using cpanm or something similar to install the module, not to mention it would most likely require a mode of 666.

Another choice would be under /var but that has the same disadvantages as the first choice.

/tmp is always good for not needing root, but is subject to being blown away on reboots (though that is not the case so much anymore.)

The problem of having mode 666 on the database files is almost always present, especially in the case of a command line application. It is simple enough to mark the app as setuid, but that introduces a set of headaches.

What is your preferred way to deal with this?

You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.

Replies are listed 'Best First'.
Re: Where to put an application database?
by davido (Cardinal) on Oct 16, 2014 at 05:25 UTC

    File::HomeDir has the my_data and my_dist_data functions, which could be a platform-agnostic way of determining where to store data.


    Dave

Re: Where to put an application database?
by BrowserUk (Patriarch) on Oct 16, 2014 at 05:52 UTC

    One of the most annoying 'features' of many Windows applications is their tendency to stuff their data files away in predefined places. (eg. APPDATA=C:\Users\UserName\AppData\Roaming\AppName\config\ and similar).

    There are two problems with this:

    1. It can take some serious searching for me to try and locate where these files are held, when (for example) I want to backup the application or transfer it somewhere; or one of these files get corrupted.
    2. It limits me to using only one configuration/set of files with that particular application, when I would often like to have two or more completely independent setups.

    Personally, I'd prefer the default location to be in (or relative to) the CWD. That way, if I wish to use the application for two or more independent projects, I can run it from those project directories and it will locate the appropriate files without my having to set environment variables or pass convoluted runtime switches to point it at the right places. I can have multiple command line sessions each in a different cwd and run multiple copies of the app without having to jump through hoops to keep them separate.


    With the rise and rise of 'Social' network sites: 'Computers are making people easier to use everyday'
    Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
    "Science is about questioning the status quo. Questioning authority".
    In the absence of evidence, opinion is indistinguishable from prejudice.
Re: Where to put an application database?
by GrandFather (Saint) on Oct 16, 2014 at 05:26 UTC

    If you are only targeting *NIX I'd be inclined to default to using ~/MyModulesDatabasePlace or some such location and allow an explicit path to be given if desired.

    If you want to be somewhat platform agnostic you might take a look at File::HomeDir and use its offerings to make a better default choice.

    Perl is the programming world's equivalent of English
Re: Where to put an application database?
by roboticus (Chancellor) on Oct 16, 2014 at 13:51 UTC

    boftx:

    To me, it depends on the "kind" of application. If the database would best used by everyone, I'd prefer a common data area (such as /var or C:/Program Data/ or equivalent. If it's an application where every user would want their own copy, I'd probably prompt the user for their preferred location (I have the same aversion to applications cluttering up my personal work area with directories and files of their own). Leaving the default as the personal work area could be fine, so long as it's easy to override and/or change, so I'm not stuck with it.

    I'd avoid putting the database in the same directory with the module, though. There may be some good arguments for it, but I've not seen a case where I'd consider it. Generally, I don't back up my OS and App directories, but I *do* back up my user and data directories. I don't back up the OS and App stuff since I can simply reinstall it. But it would suck if I couldn't retain my data without specifically choosing to back up a particular App directory.

    That's my $0.02, take it for what it's worth...

    ...roboticus

    When your only tool is a hammer, all problems look like your thumb.

      I agree with it depending upon the "kind of app" as you put it. Since this could be used by a number of different users I m leaning towards using /var as the data dir.

      The bigger issue, in my mind, is having the database files be mode 666 maybe. I know that Oracle, MySQL and others are setuid to get around that, but I need to check the docs for SQLite to see if it does that, too.

      You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.

        boftx:

        A lot of databases handle the permissions issue by having a separate server process that actually does the database work, and all clients just submit requests to that process to get the work done.

        SQLite, though, doesn't use a server process. Instead every client has a copy of the code to perform database updates. In that case, the file will need to be modifiable by everyone if you want them to be able to make any changes.

        ...roboticus

        When your only tool is a hammer, all problems look like your thumb.

Re: Where to put an application database?
by skx (Parson) on Oct 16, 2014 at 16:27 UTC

    I wrote a blog-compiler which imports a series of blog-posts, from plain-text files, into an SQLite database.

    The database lives in the home directory of the current user. That makes sense for me, but if it doesn't the user can specify it via a command-line argument.

    (App::Chronicle for reference.)

    Beyond that it depends on the use-case for the database. Is it supposed to be updated? Will things function if it is deleted/not-present?

    Steve
    --
Re: Where to put an application database?
by DrHyde (Prior) on Oct 17, 2014 at 11:12 UTC

    If you think that stuff getting blown away from /tmp isn't the case so much any more then you're in for a nasty surprise.

    However, if your code isn't running as root, the logical place to put its data is in the application's home directory. It's fairly common for significant applications to have their own user, which has a home directory somewhere outside /home. For example my mail sewer runs as ...

    Debian-exim:x:105:105::/var/spool/exim4:/bin/false

    Note the home directory. Obviously this sort of thing has to be set up by root, but the application need not run as root.

Re: Where to put an application database?
by opitzs (Sexton) on Oct 20, 2014 at 12:07 UTC
    /tmp is always good for not needing root, but is subject to being blown away on reboots (though that is not the case so much anymore.)

    Most of our servers have their /tmp directory in RAM, never use /tmp for permanent storage, the name is not just for fun...

Re: Where to put an application database?
by thargas (Deacon) on Oct 20, 2014 at 12:55 UTC

    It depends on what this application uses the database for.

    If you're using the database for system-wide configuration of the module, then putting the database with the module makes some sense, but I get the impression that this is not the case.

    If the application uses a single database for all the users of the database, then sqlite isn't really an appropriate choice. For full-blown databases, they'll be running as a different process which will have access to the database files and that problem goes away.

    If the application uses a separate database for each user, then sqlite might be appropriate, but you should allow the user to specify the database location themselves via command-line switch or environment variable or config-file or ...

    Since perl has DBI, unless you're using sqlite-specific features, I'd seriously consider simply letting the user specify database connection info (dsn, user, password, attrs) and, again, the problem goes away.

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: perlquestion [id://1104002]
Approved by GrandFather
Front-paged by toolic
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others about the Monastery: (3)
As of 2024-04-25 19:34 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found