http://qs321.pair.com?node_id=797104


in reply to Re^2: Insert BLOB into DB without using system memory, but using filehandle.
in thread Insert BLOB into DB without using system memory, but using filehandle.

Repeat after me:

  1. I will not store files inside of the database.
  2. I will not store files inside of the database.
  3. I will not store files inside of the database.
  4. I will not store files inside of the database.
  5. I will not store files inside of the database.
  6. I will not store files inside of the database.
  7. I will not store files inside of the database.
  8. I will not store files inside of the database.

Storing files inside of the database is just a Bad Idea. You are much better off doing something like the following:

  1. Insert a new record into the database (filename, size, source, date, etc).
  2. Get the ID of the new record.
  3. Take the md5 hash signature of the ID + some secret code.
  4. Your hash is something like 0123456789abcdef0123456789abcdef
  5. Update the database record to show "hash = '0123456789abcdef0123456789abcdef'"
  6. Make folder /var/media/01/23/45
  7. Write file contents to /var/media/01/23/45/0123456789abcdef0123456789abcdef

NOTE: /var/media/ could be a local volume or a remote-mounted SAN. You choose.

Now, you can partition the files across multiple servers. You can store the files on a SAN and have (many) dummy file servers in front of it.

This is (basically) how wikimedia does it. And just look at how many files they have. Storing the files inside of the database is just Not How It's Done.

Replies are listed 'Best First'.
Re^4: Insert BLOB into DB without using system memory, but using filehandle.
by songahji (Friar) on Sep 24, 2009 at 14:24 UTC
    Your idea is better, I recon. I wish I am the boss, but gotta do what the boss want. Cheers!
[reply]
Re^4: Insert BLOB into DB without using system memory, but using filehandle.
by Anonymous Monk on Dec 31, 2015 at 18:25 UTC
    Wow. Such arrogance.
[reply]

In Section Seekers of Perl Wisdom