"Sorry, I meant that the Perl module comes with installing ImageMagick itself if you do it from source."

This is not true. As documented a default install does not build Image::Magick. You can see problems just by looking at the test matrix and open bugs. See also Cannot compile Image::Magick.