some of my files conflicted with system ones

Of the various things you wrote, I'd identify this as a central issue. I would suggest putting all your modules in their own namespace, for example MyProject:: or MyCompany:: - see also On The Naming of Modules, in particular, if these are modules that are always going to remain local and that you're not going to put on CPAN, you might also consider Local::.

And is there a standard place (in my home directory) to put the directories containing the library files?

There's More Than One Way To Do It... since you say it's just per-user, then really any place in your $HOME works, and using PERL5LIB set up in a ~/.bashrc or ~/.profile is fine (keeping in mind that you might want to append to the environment variables it if they're already set from elsewhere).

You might have a look at local::lib, which will add ~/perl5/lib/perl5 to PERL5LIB by default. However, personally, I consider ~/perl5/lib a directory I can just clobber anytime so I can re-install everything from CPAN - so in fact, you might want to consider setting up a really basic Makefile.PL for your modules, such that you can make use of the default Perl installation mechanisms. See ExtUtils::MakeMaker - for example, consider developing your modules in some directory like say ~/dev/MyModule (which you can add to a version control system), which could have a directory layout like:

~/dev/MyModule/Makefile.PL
~/dev/MyModule/lib/Local/Math/EM.pm    # Local::Math::EM
~/dev/MyModule/lib/Local/Math/Levy.pm  # Local::Math::Levy
~/dev/MyModule/lib/Local/Math/Log2.pm  # Local::Math::Log2
~/dev/MyModule/bin/script.pl
~/dev/MyModule/t/*.t                   # (optional but recommended)
[download]

And a Makefile.PL as simple as:

use ExtUtils::MakeMaker;
WriteMakefile(
    NAME      => 'Local::Math',
    VERSION   => '0.01',
    EXE_FILES => [ 'bin/script.pl' ],
);
[download]

Then, in ~/dev/MyModule/, you can just do the usual perl Makefile.PL, make, make test, make install, and with local::lib set up properly, it'll all just work; plus it makes it easy for you to expand your "distribution" with more tests, and it'll be easier to grow into a "more proper" distribution, like declaring dependencies and adding more tests. Also distributing to other machines will be as simple as:

make distclean    # if you've previously run "perl Makefile.PL"
perl Makefile.PL
make manifest
make dist         # will create a Local-Math-0.01.tar.gz for you!
[download]

I think your idea of keeping your configuration files separately is a good one, since that's not something you'd want to keep in a "distribution" like the above. You could also have a look at the Config:: namespace on CPAN for plenty of pre-made solutions.

A related question is where to put executable perl programs that are specific to my login. Is there a "bin" directory position within my home directory that's seen as standard or typical?

~/bin is added to PATH as part of the default ~/.profile on Debian-based systems, that's usually what I use for site-local stuff (Update: I probably should have said "user-local" stuff). Or, local::lib sets up ~/perl5/bin, which with the above suggestion will also "just work".