I only need two things from the API - to be able to store a JSON string, and to be able to retrieve the most recent JSON string stored.

Presumably the views are accessed using some name or number.

It'd be pretty hard to beat your file system for performance for something as simple as this. Each view is simply an appropriately named file within a directory.

Have the background process write the json file to a temp directory, and then, when complete, delete the 'live' file and rename the new one into the live directory.

The foreground process simply slurps the named view from its file in the 'live' directory, and uses a (say) 1/10th second sleep-before-retry, if the file doesn't exist at the moment it tries to slurp it.

Assuming a relatively small (low hundreds or less) number of views; and relatively small files (say a few kb); any half decent file system will do a pretty good job of keeping the 'hot' views in cache and will take care of the 'refresh on write' situation automatically.

For such a simple use-case, anything else is pretty much overkill.

Thank you. You're right - I was overthinking the whole thing. I'll update the files using File::Temp's tempfile and rename, and I'll store them under the same dir, with file name corresponding to the view name.

Then, on the frontend, I'll just have a handler associated with the path '/data/viewname' that returns the viewname file contents (JSON).

There is a blurt_atomic sub in Sysadm::Install that handles the tempfile/rename routine, I'll either roll my own in a similar way, or just use the module as an added dependency.

Again, thank you, both for the time, and the good advice.

- Luke

As long as the back-end is a single machine and not a load-balanced farm *and* the number of saved files is relatively small, this will work fine. A shared filesystems across a farm presents it's own set of challenges as does a large number of files in one directory. For load-balanced farms, I would pin the service to one machine and if the number of files is expected to be greater than 10K, I would use some type of directory partitioning scheme to keep the number of files in a directory low (for those times when you need to manually read the dir).

-derby

If running a big RDBMS is feasible: PostgreSQL has pretty good json support, extended in the upcoming v9.4.

PostgreSQL docs: JSON datatypes (links to 9.4, but 9.3 links are easily reached from there)

postgres/mongodb JSON comparison (EnterpriseDB is a postgres support company)

Thanks for the reply. It's definitely feasible, but I'm looking for a simple solution. As I said, there are no relations between the data other than time precedence.

- Luke

This is a good suggestion because of the flexibility and depending on usage, RAM, and strategy choices seems likely to me to be much faster than file based stuff. If it’s a few hundred smallish JSON “views,” it could all live in memory and only need IO on cache misses/expirations. If it’s mostly needing IO then it would be slower since it’s just one more layer of housekeeping on top of it all. As you say, the CHI backends make trying it out in different ways quite easy.