japhy,

I have experienced similar situations, needing to process/strip the contents from another web site. In my case it was getting info from an "internal" site to be displayed on an "external" site.

In our case, I also wrote the site being called and reformatted, so I was able to put markers in the source output that I could easily strip with a straightforward regex. I didn't need to get it a line at a time, so I didn't need to re-write any LWP type functionality, so I got off easy.

In another similar case, I used a behind-the-firewall server to produce delimited data from an internal financials database (thus no absolutely access from the 'outside' allowed). Again I used markers to isolate the customer data, then created a page from that. Luckily, it was a single row of data, so I didn't need to process by line ( although I could have split on "yet another set of markers". So while my situation differed a bit from yours, I think you did the right thing...which is you solved a problem in a way that you were able to using your own skills and insight.

I don't think that you should be worried about either doing this or posting that you did this. I suspect your customer (even if it's just your own project, thus you) is looking for end results. And while I don't subscribe to the "end justified the means", I do subscribe to TIMTOWTDI.

thanks for sharing your experience, and posting useful code.