hmm..

s|&#(\d+);|pack("c",$1)|ge
[download]

ought to work but that produces a bunch of control characters. so there's obviously a subtlety i've missed