.t files are perl files. They have a different suffix to identify the fact they are tests, not production code.

When I "use" Test::Simple, Test::More, etc., in a .pl file and run it with Perl it produces test results. Likewise, running that same .pl file with prove will produce test results although the output appears different. What's the advantage of using prove? Why not simply use Perl?

We do use perl. The different extension simply indicates to the user or test harness (e.g. prove) that this is a test which should be run. Prove provides a useful summary of the results of running many test scripts. The test script output (TAP - test anything protocol) is human-readable, but more easily summarised by a program. That's what prove is. (The volume of test data can be large, a summary is generally more useful unless you're hunting specific failures).

Are tests embedded into production modules? If not, how does one test (verify) that the act of physically removing tests from production code didn't accidentally interject some errors?

There are modules for embedding tests, e.g. Test::Inline, but normally the production code is left alone. The production code is normally included into the the .t files via use, so the .t file can call exactly the same code which would run in production.

I understand that the use of t/ directories to contain .t files is a convention for normal module installation. My problem is that I am coming from the Win32 environment using the ActiveState installation and I therefore rely on ppm for my installs. I don't get to do the "make test" part of the installation so perhaps I just don't have a full appreciation of what testing does. But I do want to have that understanding as I am working on some code I would like to distribute. I am sure development will go better if I can embed testing in it.

Automated testing is a important and very useful to check you don't break existing code or re-introduce old bugs. Some people go overboard with it, always beware being over-zealous about a methodology or programming practice. But my opinion is that it is very useful, if for no other reason than when you're writing a module (and hence designing an API) it gives you experience in writing code which uses that API - which often leads to a rethinking and refinement process.

'make test' pretty much just runs prove, so you're not missing out on too much there.