I think the idea behind that is that you always know where the exit from the function occurs.

The idea comes from "structured programming". Dijkstra's famous "Go To Considered Harmful" isn't about avoiding the goto keyword. It's about having structured units in your program (what we now call blocks), each with a single entry and exit point. Goto defeats that, but next, last, redo, and multiple return statements do so as well.

Structured programming is considered useful is when you are make statements (proofs) about the program itself. Proofs typically make use of pre- and post conditions. Not having single entry/exit points make those proofs much harder, if not impossible.