I have a large array that contains sets of integers.
array[0]="1,100,105,1003";
array[1]="42,314,2718,30302";
etc...
I would like to save RAM with a more efficient representation
of these sets. Here are a few facts about the numbers:
- The numbers do not repeat
- The numbers represent an index into another array
- I can reorder the data if I really need to
- Some numbers are much more common than others
- I don't care much about how long it takes to compress the data
- I would like to decompress the data relatively quickly
- In some cases there are thousands of entries per row,
other cases only one.
The best answer might be to just use a database. I was
thinking about a RAM-based solution in an attempt to get
blazing speed in a mod_perl application. It would be
nice if I could look up and decode about 100 of these rows
in 0.05 seconds or so.
I would also like to be able to load these numbers into
RAM relatively quickly, so I would like a data structure
that can be stored and restored with Storable or some other quick
method.
I am looking for ideas, pointers to algorithms, modules,
or code.
It seems like a fairly fundamental area of computer
science, but I don't know what this area is called.
Thanks!
It should work perfectly the first time! - toma
-
Are you posting in the right place? Check out Where do I post X? to know for sure.
-
Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
<code> <a> <b> <big>
<blockquote> <br /> <dd>
<dl> <dt> <em> <font>
<h1> <h2> <h3> <h4>
<h5> <h6> <hr /> <i>
<li> <nbsp> <ol> <p>
<small> <strike> <strong>
<sub> <sup> <table>
<td> <th> <tr> <tt>
<u> <ul>
-
Snippets of code should be wrapped in
<code> tags not
<pre> tags. In fact, <pre>
tags should generally be avoided. If they must
be used, extreme care should be
taken to ensure that their contents do not
have long lines (<70 chars), in order to prevent
horizontal scrolling (and possible janitor
intervention).
-
Want more info? How to link
or How to display code and escape characters
are good places to start.
|