Test::More doesn't care about the organization of your test files. Test::Harness doesn't care either, as long as you pass a list of files to run to its runtests() subroutine.

Whether you're using ExtUtils::MakeMaker or Module::Build, you have the ability to specify exactly which tests to run at that stage. I usually use File::Find to look for all .t files beneath the t/ directory and its subdirectories and pass that list (or catenate it into a string) to the appropriate place.