Some thoughts... Do you anticipate needing to run anything in parallel? If so, MPI / PVM might be worth investigating. If not, things can be simpler. If all you need to do is divide up large jobs into small pieces, send them off to other systems (i.e. cluster or grid nodes) to run, then collect the results. This could result in a system with sections to:

• Take a large job and divide it into N small jobs
• A central (or distributed) system for keeping track of job status. This could be database tables and/or a central process.
• Queues for submitting jobs to nodes and taking appropriate action when they are completed. If results are stored centrally in a database, this might mean only monitoring node status and tracking errors as well as completion of jobs.

It's not impossible for one person, but the complexity can be high and debugging distributed systems can be tricky.

A commercial product called LSF (www.platform.com) does much of this and works well with cluster applications. With it, one may create queues for specific systems or sets of systems, submit jobs, and monitor them.

For grid versus cluster applications, all this gets more fuzzy. Are the grid systems shared? How reliable are they? How much redundancy is needed?

Like I said, these are some thoughts and questions. Hope it's useful. :)

---

---

• Are you posting in the right place? Check out Where do I post X? to know for sure.
• Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:

  <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>

• Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
• Want more info? How to link or How to display code and escape characters are good places to start.