This is my conclusion of it. It can be wrong :).

Well, it uses %saw hash to keep track on how many inputs have passed (NOTE the return before increment). Grep gets number when asking keep or lose. ! will reverse the ok/fail answer.

So when first of duplicate inputs gets there, $saw{$_}++ returns 0 and increments. ! will reverse that to 1 and grep takes it in. second has already a value and $saw{$_}++ returns 1 (and increments) which ! reverses to 0 and grep drops.