Beefy Boxes and Bandwidth Generously Provided by pair Networks
more useful options

comment on

( #3333=superdoc: print w/replies, xml ) Need Help??

A project I'm working on now is a system for data transformation pipelines. Kind of like a makefile for data. So your pipeline might be something like:

  • Output the following tabular data as Excel:
    • Run the following query on the following database, resulting in tabular data.
      • SELECT, ... FROM ...
      • Create a temporary database with these tables in it:
        • Create a table named "person" and load it with tabular data:
          • Parse this data as CSV:
            • Download data from
        • Create a table named "companies" and load it with tabular data:
          • Parse this data as tab-delimited data:
            • Download data from

So you'd load up the pipeline, and tell the pipeline that you want the Excel file. And it thinks: OK, so I need to query this database, but first I need to build the database, how do I do that... I need to create it and load these tables... how do I do that... etc. You get the picture.

And in fact, some details of the pipeline are inferred during processing. We often don't want to hard-code file formats like CSV into the pipeline, but instead infer them from HTTP content-type headers and so forth.

It's still at a fairly early stage, but it's progressing quite nicely, and Moose roles are an important part of the implementation.

The reason being that at many stages in the process you need mix-and-match transformations. You need an object which accepts CSV and outputs Excel, or accepts tab-delimited data and outputs Excel, or accepts CSV and outputs an SQL table, or accepts an SQL table and outputs an HTML table.

OK, so each of these could be a class, but it's far easier to do as a role. When you come to a stage in processing, you instantiate a very generic object (which just has a few debugging methods, a unique identifier string, a link back to the pipeline object, etc) and compose it with the roles it needs to be able to handle (read CSV, write Excel).

The technique which you showed in your example is to make a subclass of Math::BigInt, and stuff the new methods into that. It's worth noting that actually, this is exactly how Moose role composition works internally - it creates a new class for you, and reblesses the object into that class. But it's all done on the fly at run-time, so you don't need to create modules for each class, choose names for them, etc. All that is abstracted away.

perl -E'sub Monkey::do{say$_,for@_,do{($monkey=[caller(0)]->[3])=~s{::}{ }and$monkey}}"Monkey say"->Monkey::do'

In reply to Re^3: Altering the inheritance path of an object by tobyink
in thread Altering the inheritance path of an object by citromatik

Use:  <p> text here (a paragraph) </p>
and:  <code> code here </code>
to format your post; it's "PerlMonks-approved HTML":

  • Are you posting in the right place? Check out Where do I post X? to know for sure.
  • Posts may use any of the Perl Monks Approved HTML tags. Currently these include the following:
    <code> <a> <b> <big> <blockquote> <br /> <dd> <dl> <dt> <em> <font> <h1> <h2> <h3> <h4> <h5> <h6> <hr /> <i> <li> <nbsp> <ol> <p> <small> <strike> <strong> <sub> <sup> <table> <td> <th> <tr> <tt> <u> <ul>
  • Snippets of code should be wrapped in <code> tags not <pre> tags. In fact, <pre> tags should generally be avoided. If they must be used, extreme care should be taken to ensure that their contents do not have long lines (<70 chars), in order to prevent horizontal scrolling (and possible janitor intervention).
  • Want more info? How to link or How to display code and escape characters are good places to start.
Log In?

What's my password?
Create A New User
Domain Nodelet?
and the web crawler heard nothing...

How do I use this?Last hourOther CB clients
Other Users?
Others romping around the Monastery: (7)
As of 2023-11-30 14:21 GMT
Find Nodes?
    Voting Booth?

    No recent polls found