I have counted on this behavior for years. Your question has made me realize that it is, in fact, not documented anywhere obvious. I think I picked up the habit because of the examples given in the Test::More documentation for BAIL_OUT and diag.

Now, Test::More is implemented in terms of Test::Builder, and the latter's documentation does say that all tests return true if the test succeeds and false if it fails. So the de facto answer appears to be "yes, you can." But the paper trail seems to be lacking, so I would say the de jure answer is "no, you can not." Maybe a documentation bug report should be filed?