http://qs321.pair.com?node_id=120177

I feel something is missing in the open source world: a web browser. There are Mozilla and w3m, but the coolest ones continue to be IE and Opera. Why? Is it so difficult to develop a web browser? What can be done from within the Perl community to build a web browser superior to IE and Opera? Has anyone thought about it?

Replies are listed 'Best First'.
(ichimunki) Re: Perl Web Browser?
by ichimunki (Priest) on Oct 20, 2001 at 07:39 UTC
    I swear I've written on this topic recently, but all I can find is this node about Tk::HTML.

    Other browsers in Perl:
    artemis: this is a total hack job, and the lunatic that started the project can never seem to keep make progress. I'm guessing there's a major rewrite and some architectural rethinking in Artemis' near future. (If the author manages to pay any attention to his projects, that is, he's a real slouch in that regard.)

    Whole book on the topic of web client programming online at O'Reilly.

    A WML browser: not quite HTML, but might offer some architectural ideas.

    Of course tkweb is included with Tk::HTML. Farther along than some stuff I've seen, but I think it has a ways to go. Table-based sites like PM don't render at all.

    And I have a project that I found one day on the web, that I can't find a link to. Note to developers: always put at least your name on a script, and maybe a link or an email address.

    But the real question: what can a Perl browser offer than one written in C (Mozilla) and one written in C++ (Konqueror) can't? Both the Mozilla and Konqueror browsers are based on rendering engines that are theoretically separable from much of the other software included in a browser (AFAIK), so it might be better to build bindings that could link Perl to those rendering engines, or to work on Perl scripted plug-ins.

    Different people have very different ideas about what web browsers are and how they should behave. On the one hand a browser has to render every pixel exactly as the designer intended, even though many web designers are incredibly sloppy about the way they build pages, so that clients have to guess what they meant to do. On the other hand, some people think a browser that displays the text mostly correctly is sufficient-- they won't use sites that don't conform well enough to the standard to allow for that.

    And in the end, a Perl browser comes down to a question of making choices. You could do everything, but you'd need a super-computer to run it on, Perl trying to emulate IE6 would run slooooow on a normal home computer. OTOH, a Perl browser that targeted a specific application would make a lot of sense. And it might help to think of it as an HTTP/HTML client more than a "browser". That way you're freer to think outside the norm.

    I think Tk::HTML and tkweb show the most promise, and would recommend looking at ways to help push those forward, or merge the two-- since tkweb does not use Tk::HTML but Tk::Text (which is a very versatile widget in the first place).
Re: Perl Web Browser?
by jryan (Vicar) on Oct 20, 2001 at 03:57 UTC
    Of course people have thought about it; just off the top of my head I can think of TK::HTML. However, the module is EXTREMLEY hard to use. One of the reasons many people are discouraged from creating open source web browsers is because of the sheer difficulty in parsing bad html. One of the key things IE/Mozilla/Opera have going for them is their ability to parse extremely bad html markup into decent looking pages. Parsing html isn't easy; anyone who has tried without using HTML::Parser will tell you that. While I think a decent one can be written in perl, it would be much larger than a one person project; it would take a community effort (probably a sourceforged one) to be have the manpower to do it.
      It could have the following feature: automatically email the webmaster with syntax error report.

        That would make the guys at geocities and other 'free' web servers extremely happy. :-)

        f--k the world!!!!
        /dev/world has reached maximal mount count, check forced.

        The problem is though, that almost all html on the internet is bad html. Even sites you'd expect to be better use horrible code.

        PerlMonks, Microsoft, and Mozilla all fail validation. And they're the only 3 I've tried. This is because writing correct w3c validated html which renders well in all browsers is extremely difficult without losing page complexity. These problems are increased on dynamically generated pages.

        To avoid these problems, the w3c is pushing the xhtml standard; which is similar to HTML 4.01 except has to be structured in xml style syntax.

Re: Perl Web Browser?
by toma (Vicar) on Oct 21, 2001 at 01:55 UTC
    I tried to write a text-based perl web browser called PROWL: Perl->Request->Object->Web->Link().

    My goal was to create a web browser to help automate the generation of web data mining code.

    I used these modules:

    The biggest challenge was getting the HTML layout to work. I started to get tables to work, but I gave up when I ran into frames. If you ignore these issues, it is easy to write a browser.

    To make the HTML easy to parse, I suggest Dave Ragget's tidy program. Tidy converts hard-to-parse HTML into easy-to-parse XHTML. A perl module to interface to the tidy C code would be useful. In addition to being helpful for writing a perl web browser, a tidy perl module would be useful for web site maintenance and data mining.

    It should work perfectly the first time! - toma

Re: Perl Web Browser?
by sm3g (Hermit) on Oct 20, 2001 at 06:10 UTC
    There is a small one that comes with the perl/Gtk mail client Pronto! (appropriately called prontobrowser.) They use it for the help system but I believe it can also be used to browse the web in general.
    Unfortunately I haven't been able to get any of the HTML widgets it uses (CscHTML, GtkHTML, or GtkXmHTML) to 'make' and install so I can't give any more firsthand information. If it is as well done as Pronto! is, it's worth a look.
    sm3g
Re: Perl Web Browser?
by BlueLines (Hermit) on Oct 25, 2001 at 01:45 UTC
    Ok, some general comments / replies to everything that's posted here:
    • Mozilla is written in c++, not c.
    • gtkhtml is the rendering library (most) gnome apps use as a help browser. And its' "real" html support is heinous
    • perl is a great language, but it's not nearly fast enough to be used for an app as complex as a web browser. and i mean a graphical web browsers, not lynx. i have a 1.4ghz athlon, and the slowest app in both windows and linux is still my web browser (ie6 and mozilla 0.9.5).

    there is the gecko library (the rendering engine from mozilla) that is freely available. perhaps a gtk frontend could be written in perl. but the galeon people have already done a pretty good job of this in C; if your goal is to make a faster web browser, perl isn't really the right way to go.

    and i'm not sure what you mean by the "coolest ones". everyone in my office drools over the tabbed interface mozilla 0.9.5 has, while ie continues to have scripting issues (and won't recognize urlencoded urls as of ie6), and opera costs money.

    i should note that i'm a bit biased; i've been using mozilla for the last 2 years (exclusively for the past 16 months). i've submitted and worked on alot of userland bugs and compiler related issues. and to be perfectly honest, i do more web browsing at work with nc and telnet than i do with any browser....



    BlueLines

    Disclaimer: This post may contain inaccurate information, be habit forming, cause atomic warfare between peaceful countries, speed up male pattern baldness, interfere with your cable reception, exile you from certain third world countries, ruin your marriage, and generally spoil your day. No batteries included, no strings attached, your mileage may vary.
Re: Perl Web Browser?
by Elvis (Sexton) on Oct 23, 2001 at 23:15 UTC

    This comment is slight OT, but Netscape 6 is a *VAST* improvement over netscape 4.77 for *nix. (I haven't had to kill -9 it once!) I'm gusssing your experience of netscape 4 has made you leave out netscape in the list of 'coolest ones'. :)

    Still, it would be nice to have a good open source web browser. I think it's likely that mozilla will make the grade soon. BTW, there is GNUNavigator which is a browser bolt-on for emacs (IIRC). I've no idea what that's like though.

    Also, what about konqueror? The latest version has better CSS 2 support than any other browser, (that's including IE 6) IIRC.

Re: Perl Web Browser?
by fr3ez (Acolyte) on Oct 25, 2001 at 12:12 UTC
    Some excellent answers here already, but one thing no-one has pointed out is the whole standards issue.
    I know at least the GIF standard is copyrighted by Compuserve and while I'm not a lawyer I'm sure there'd be some sort of royalty fees or something involved.
    While yes you could just support other graphic formats but people use gifs and jpgs most commonly.
    One other not mentioned so far is lynx, yes not perl solution or a graphical browser but free and does a good job of html, but I personally only use it as a scrape by measure, because I've come to rely on graphics.
    I'm sure a lot of other standards have thorny legal issues as well like Java and VBScript support, and these days you need that sort of stuff in a browser because people's sites use such technologies (as much as it pains me).

    -That is all
      If I recall correctly, the GIF licensing issues only affect programs that attempt to create GIFs, not render them.

      Oops! Apparently I spoke too soon. A quick web search revealed that it is obviously *much* more complex than that.
Re: Perl Web Browser?
by archen (Pilgrim) on Oct 26, 2001 at 02:27 UTC
    yeah, I think it comes down to the "right tool for the right job". Perl is the only language I even program in now (other than the occasional VB hack I have to do at work), I love Perl, but Perl is a BAD choice for a web browser. The speed would obviously be horrible, but there are other things to consider as well (maintainability issues).

    Keep in mind that Mozilla IS open source. You're free to fork the code and try doing it your own way (something like the guys at kmeleon are doing. Now if you ask me, what REALLY needs to be done, is some sort of Perl plug in that lets Perl scripts and Mozilla communicate. If I could get Mozilla to execute perl scripts on the fly to do certain things, I'd be a VERY happy camper. Yeah yeah, I know - security problems, but still...
Re: Perl Web Browser?
by zakzebrowski (Curate) on Oct 24, 2001 at 17:29 UTC
    Bah, just spend the $18 bucks for Opera, and don't worry about it... (One of my scripts for school checked to see what agent you were using... if you weren't using opera it would display an ad at the bottom of the page... :) )
    UPDATE: To the people that gave this node a --, I'm promoting code reuse here. This is not to say there shouldn't be a web browser in perl, but this is just my humble opinon... bah humbug

    ----
    Zak
Re: Perl Web Browser?
by EvanK (Chaplain) on Oct 25, 2001 at 11:34 UTC
    Well, here's a thought...is there any sort of graphical web browser type of thing one could use in conjunction with a Win32::GUI program? (aside from IE and Win32::OLE) if not, that would be an idea to consider.

    ______________________________________________
    RIP
    Douglas Noel Adams
    1952 - 2001

Re: Perl Web Browser?
by Anonymous Monk on Oct 27, 2001 at 04:07 UTC
    What about Galeon? I've found it fast, reliable and easy on the eyes. It's basic and does the job, but you need a running build of Mozilla for it to work.

    -doonyakka
      Used to be that whenever I dropped back into windows for compatibility reasons I would realize how much better IE was than mozilla or netscape. But now that I have Galeon I feel hampered in IE. Sort of wish there was a way to get the Gecko rendering engine w/out Mozilla though.
Re: Perl Web Browser?
by Draxil (Novice) on Nov 08, 2001 at 02:21 UTC
    Mozilla inferior to IE??? Thats a matter of taste. But to say that Galeon, whatever the kde mozilla frontend is called and Konquorer are inferior to IE is just plain lying (IMO). Also in the open source arena are links and lynx, which are in a different arena but still rock.

    Still berating parts of your argument aside (and mabye getting back on topic), a perl web brower could rock. If for no other reason than sheer quick hackability :)
    "blue berry muffins are not just a snack.... ..they are a way of life"