... use a Makefile.PL ...

This is really the best option, and it's not hard once you figure out which of ExtUtils::MakeMaker's many options to tweak:

use ExtUtils::MakeMaker;
WriteMakefile(
        NAME    => 'YourModule',
        OBJECT  => 'YourModule_wrap.o',
        # Add extra include paths here, if needed
#        INC     => '-I/path/to/include',
        # Add shared and static libraries here, if needed
#        LDFROM  => 'YourModule_wrap.o -L/path/to/lib -lyourmodule',
);
[download]

Basically, you add Foo_wrap.o to OBJECT, and possibly modify INC and LDFROM if you're compiling against a C library. SWIG could/should even generate a basic Makefile.PL when it generates the wrapper code.