http://qs321.pair.com?node_id=796527


in reply to Re: Insert BLOB into DB without using system memory, but using filehandle.
in thread Insert BLOB into DB without using system memory, but using filehandle.

I wish I could, but the data must be in the DB. Disk space is not an issue in this project.

Replies are listed 'Best First'.
Re^3: Insert BLOB into DB without using system memory, but using filehandle.
by jdrago_999 (Hermit) on Sep 24, 2009 at 00:26 UTC

    Repeat after me:

    1. I will not store files inside of the database.
    2. I will not store files inside of the database.
    3. I will not store files inside of the database.
    4. I will not store files inside of the database.
    5. I will not store files inside of the database.
    6. I will not store files inside of the database.
    7. I will not store files inside of the database.
    8. I will not store files inside of the database.

    Storing files inside of the database is just a Bad Idea. You are much better off doing something like the following:

    1. Insert a new record into the database (filename, size, source, date, etc).
    2. Get the ID of the new record.
    3. Take the md5 hash signature of the ID + some secret code.
    4. Your hash is something like 0123456789abcdef0123456789abcdef
    5. Update the database record to show "hash = '0123456789abcdef0123456789abcdef'"
    6. Make folder /var/media/01/23/45
    7. Write file contents to /var/media/01/23/45/0123456789abcdef0123456789abcdef

    NOTE: /var/media/ could be a local volume or a remote-mounted SAN. You choose.

    Now, you can partition the files across multiple servers. You can store the files on a SAN and have (many) dummy file servers in front of it.

    This is (basically) how wikimedia does it. And just look at how many files they have. Storing the files inside of the database is just Not How It's Done.

      Your idea is better, I recon. I wish I am the boss, but gotta do what the boss want. Cheers!
      Wow. Such arrogance.