From the pod:

#!/usr/bin/perl -w
use HTML::Scrubber;
use strict;

my $html = q[
<style type="text/css"> BAD { background: #666; color: #666;} </style>
<script language="javascript"> alert("Hello, I am EVIL!");
</script>
<HR>
    a   => <a href=1>link </a>
    br  => <br>
    b   => <B> bold </B>
    u   => <U> UNDERLINE </U>
];

my $scrubber = HTML::Scrubber->new( allow => [ qw[ p b i u hr br ] ] )
+;
print $scrubber->scrub($html);

$scrubber->deny( qw[ p b i u hr br ] );
print $scrubber->scrub($html);
__END__
Output:
<hr>
    a   =&gt; link
    br  =&gt; <br>
    b   =&gt; <b> bold </b>
    u   =&gt; <u> UNDERLINE </u>

    a   =&gt; link
    br  =&gt;
    b   =&gt;  bold
    u   =&gt;  UNDERLINE
[download]

I use HTML::Scrubber on one of my sites, the only problem I have with it (which I was vaguely thinking of posting as a new question only yesterday) is that I see no way to enforce attribute inclusion.

Say the user submits:

<a href="http://example.com">text</a>

I would like to automatically insert, or mandate, the xrel="nofollow" attribute and value - I can't see a simple way of doing this short of re-using the HTML::Parser, or a fragile regexp.

That's the only shortcoming I see with HTML::Scrubber.

Steve
---
steve.org.uk

Have you considered subclassing HTML::Scrubber? Below, I inject the xrel attribute into each anchor before validation.

$ cat XREL.pm
package XREL;
use strict;
use base 'HTML::Scrubber';

sub _validate {
    my ($self, $t, $r, $a, $as) = @_;

    if ( $t eq 'a' ) {
        $$a{ rel } = 'nofollow';
        push @$as, 'rel' unless grep { /rel/ } @$as;
    }
    $self->SUPER::_validate( $t, $r, $a, $as );
}

1;
[download]

$ cat scrub.pl
#!/usr/bin/perl

use warnings;
use strict;
use XREL;

my $scrubber = XREL->new( allow => [ qw[ a p b i u hr br ] ] );
$scrubber->rules(
        a => {
            href => 1,
            rel => qr/^nofollow$/i,
            '*'  => 0,
        }
);

my $html = q[<a href="http://perlmonks.org">link </a>];
print $scrubber->scrub($html), $/;
$html = q[<a href="http://perlmonks.org" rel="nofollow">link </a>];
print $scrubber->scrub($html), $/;
$html = q[<a href="http://perlmonks.org" rel="xxx">link </a>];
print $scrubber->scrub($html), $/;
$html = q[<a href="http://perlmonks.org" rel="xnofollow">link </a>];
print $scrubber->scrub($html), $/;

__END__
output:
<a href="http://perlmonks.org" rel="nofollow">link </a>
<a href="http://perlmonks.org" rel="nofollow">link </a>
<a href="http://perlmonks.org" rel="nofollow">link </a>
<a href="http://perlmonks.org" rel="nofollow">link </a>
[download]

update:changed xrel="nofollow" to rel="nofollow"

Off the top of my head, I would probably throw the thing at HTML::Parser, and check for any tag that doesn't match your list of acceptable tags. Any bad tags would prompt an error and refusal to accept.

It does, like any other detainting procedure, need to use a list of acceptable tags, rather than a list of unacceptable tags. It's much easier to add to a list of acceptable tags ("Hey, my favourite tag, dl, isn't working! Add it!") than to maintain a list of unacceptable tags ("Darn, look what this idiot just did!").

Option 2: grab the source to Everything, and steal it. ;-)

There are some arguments against that. All of that stuff takes effort (the warning) and that effort could have gone into making the system safer. You also need to consider the aptitude of the users. Are they likely to accidentally make mistakes in input? If so, the validation has another function which is making the system more usable.

So I guess I'm saying there are cases where you can leave it wide open, but it's a small subset of cases.

That leads to your other question, which is are there some subsets of acceptable tags that people have already defined? I'm very interested in the answers because I think you're right that this problem gets solved repeatedly all the time.

Do the same thing that most wikis use and create a new notation with well defined rules. Then convert that to HTML. Assuming your new notation is well designed that would be the end of the problem. HTML is not suited to this as it has too many edge cases where things can happen that you may not anticipate, and filtering those constructs out is difficult if you intend to allow some of them to pass through, requiring you to handle parsing html, which is expensive.

Also if go with HTML-like markup you need to consider carefully where you handle task like filtering. Putting on the submit has different ramifications from putting at the fetch.

1. It works.
2. Your users will probably already be familiar with a similar system.
3. You will have complete control over the tags, including the ability to create custom tags that combine multiple HTML elements.

• PHPBB's
• vBulletin's

I'm not sure I understand why. You could do what you say is necessary by providing a dhtml wysiwig html editor for example, certainly you don't need full xhtml to do the simple things you mention. Or maybe there is something else?

Also I have not used it but you probably know Bricolage and other CMS solutions, at any rate this seems like the kind of thing where either you use a very elegant solution, skip providing a solution, or end up making one and tweaking it forever. Anyway, a "listing" page doesn't need more than maybe boldface/italics and hrefs, no?

The modules available to filter HTML -- HTML::Scrubber, HTML::Sanitizer -- are missing what, to me, is a very important feature: enforcing tag balance. Those of us writing portal-style snippet-editing stuff often need to make sure that open tags match close tags, at the very least. Ideally, proper nesting would be enforced (and munged in by the filter itself).

Tag balance and proper nesting matter for two reasons: browsers often do odd, ugly and non-intuitive things to layout in the presence of unbalanced tags; and it is sometimes convenient to store snippets in x(ht)ml contexts, and forcing proper tag semantics on input removes the need to escape data in these kinds of systems. I have a hacked-up filter/balancer based on HTML::TreeBuilder. It works okay, is in the XML::Comma svn repository, and could be released with our next production version. But it would perhaps be nice to not impose yet-another-HTML-filter on the world, and to extend one of the existing offerings in this direction. So my question is, why don't other folks seem to care about this feature as much as we do -- what are we missing?

I do notice that perlmonks does some auto-balancing in posts (although not quite the same way I would). I did a bit of poking around at the Everything site, and it looks like the filter modules are not separate CPAN entities.

Tag balancing is a feature (requirement) of XML, not HTML. You want an XML module if you are parsing XML, but if you're parsing HTML (as in, the HTML that you may get from an arbitrary site over which you have no control), you need the HTML modules.

If your documents are well-formed, then I would suggest XML::Twig. I use XML::Twig for doing things such as taking HTML tables, copying the header, and reinserting it every 5th or 10th or whatever-th row such that, in long tables, you don't need to go back to the top of the screen to find it. And to alternate background colours on rows (setting the class attribute to "odd" or "even", and letting CSS actually do the colouring).

But, if a user over which you have no control will send you text in a form, you're probably better off assuming that they may not balance their tags.