Just another idea. Maybe it is possible to do it by existing tools, but I just did not figured yet how to.

Update Just found it... :-(

It is "/msg me when reply..."

I believe it makes sense to be a default, right?

pmas