I am writing a couple of web-page-scraping tools that will help me in my job seek. I already have something working, but what I am missing is a nice pure perl solution that would format a web page to a nice plain text, so that if an announcement is, for any reason, removed, I still have a chance of getting to the contents
And hence the question: is there anything like lynx -dump in Perl? I dug into CPAN for about half an hour and tried html2text, but it didn't really do a good job...
For the few of you that don't know what lynx is and what it does:
lynx - a general purpose distributed information browser for
the World Wide Web
Lynx is a fully-featured World Wide Web (WWW) client for
users running cursor-addressable, character-cell display
devices (e.g., vt100 terminals, vt100 emulators running on
Windows 95/NT or Macintoshes, or any other "curses-oriented"
-dump dumps the formatted output of the default document or
one specified on the command line to standard output.
This can be used in the following way:
lynx -dump http://www.subir.com/lynx.html
Thanks a lot in advance for your help
In theory, there is no difference between theory and practice. In practice, there is.
Posts are HTML formatted. Put <p> </p> tags around your paragraphs. Put <code> </code> tags around your code and data!
Titles consisting of a single word are discouraged, and in most cases are disallowed outright.
Read Where should I post X? if you're not absolutely sure you're posting in the right place.
Please read these before you post! —
Posts may use any of the Perl Monks Approved HTML tags:
You may need to use entities for some characters, as follows. (Exception: Within code tags, you can put the characters literally.)
- a, abbr, b, big, blockquote, br, caption, center, col, colgroup, dd, del, div, dl, dt, em, font, h1, h2, h3, h4, h5, h6, hr, i, ins, li, ol, p, pre, readmore, small, span, spoiler, strike, strong, sub, sup, table, tbody, td, tfoot, th, thead, tr, tt, u, ul, wbr
Link using PerlMonks shortcuts! What shortcuts can I use for linking?
See Writeup Formatting Tips and other pages linked from there for more info.
| & || & |
| < || < |
| > || > |
| [ || [ |
| ] || ] ||