I have almost never seen it used anything like correctly, even when it was used inside of the core Perl distribution.

If you think you have used it correctly, then you are almost definitely wrong unless you can tell me that you know that it works differently on a croak and a confess and can tell me what the difference is. (Hint: It works differently on croak between 5.005 and 5.6. Are you still sure you used it correctly?)

I disagree. If you know where you are in a calling stack, this has worked for me numerous times:

{
    local $Carp::CarpLevel = 2;
    carp "Some error";
}
[download]

No need to worry about croak or confess here that I can see. What am I missing?

What is wrong with this?

What if someone calling your code wants to do what you are doing? They can't because no matter what they do with $Carp::CarpLevel, you are overriding them.

What if someone wants to later modify your code? They don't realize that somewhere in the guts you have hardcoded an assumption about the shape of the call-stack. So your code is fragile.

What if you are wrapping a module which might croak? If so then your callstack calculation is almost surely wrong. Read the guts of Carp to find out why.

What if you are wrapping a module which might croak and might confess? If so then there is no single way to get the right desired result.

And on and on. I gave this topic a lot of thought, and I concluded that what you are trying to do is a bad idea. The fact that many people (like you) think they know what they are doing and get it wrong consistently just convinces me of that even more.