Beefy Boxes and Bandwidth Generously Provided by pair Networks
Do you know where your variables are?
 
PerlMonks  

Re: Perl solution for storage of large number of small files

by BrowserUk (Patriarch)
on Apr 30, 2007 at 10:13 UTC ( [id://612729]=note: print w/replies, xml ) Need Help??


in reply to Perl solution for storage of large number of small files

How are the 'files' keyed?

Is lookup always by name or are there other factors involved?

What are the minimum & maximum sizes? Are these predictable or are you likely to get occasional files that break those limits?

Do you need single or multiple concurrent access to them?


Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.
"Science is about questioning the status quo. Questioning authority".
In the absence of evidence, opinion is indistinguishable from prejudice.
  • Comment on Re: Perl solution for storage of large number of small files

Replies are listed 'Best First'.
Re^2: Perl solution for storage of large number of small files
by isync (Hermit) on Apr 30, 2007 at 10:48 UTC
    32 chars names, lookup is always by name, full reads no seeks inside.

    Files can be everything from 5-500K of data...

    Concurrent writes to the storage, but never on the same file (as everything is passed through an index, serializing events, see above reply)

    Sounds like you have an interesting solution in mind...
    (This thread is becoming a fun-discussion, not only solving my problem but thinking about data-mining in general)

Log In?
Username:
Password:

What's my password?
Create A New User
Domain Nodelet?
Node Status?
node history
Node Type: note [id://612729]
help
Chatterbox?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (1)
As of 2024-04-25 19:43 GMT
Sections?
Information?
Find Nodes?
Leftovers?
    Voting Booth?

    No recent polls found