The only way I can make the tests fail in this manner are to attempt to run the tests when the current working directory is not the directory of the extracted tarball.

$ pwd
/home/chris
[canker:]$ perl foo.pl
[20:33:46] /home/chris/repos/toolchain/Archive-Extract/t/01_Archive-Ex
+tract.t .. 
ok 1 - use Archive::Extract;
ok 2 - Archive::Extract->can('types')
ok 3 -    Got a list of types
# 01_Archive-Extract.t
Key 'archive' (01_Archive-Extract.t) is of invalid type for 'Archive::
+Extract::new' provided by ANON at /home/chris/repos/toolchain/Archive
+-Extract/t/01_Archive-Extract.t line 264
not ok 4 -    Object created based on 'tgz'

#   Failed test '   Object created based on 'tgz''
#   at /home/chris/repos/toolchain/Archive-Extract/t/01_Archive-Extrac
+t.t line 265.
Can't call method "error" on an undefined value at /home/chris/repos/t
+oolchain/Archive-Extract/t/01_Archive-Extract.t line 266.
1..4
# Looks like you failed 1 test of 4.
# Looks like your test exited with 2 just after 4.
Dubious, test returned 2 (wstat 512, 0x200)
Failed 1/4 subtests 
[20:33:47]

Test Summary Report
-------------------
/home/chris/repos/toolchain/Archive-Extract/t/01_Archive-Extract.t (Ws
+tat: 512 Tests: 4 Failed: 1)
  Failed test:  4
  Non-zero exit status: 2
Files=1, Tests=4,  1 wallclock secs ( 0.12 usr  0.03 sys +  0.83 cusr 
+ 0.09 csys =  1.07 CPU)
Result: FAIL
[canker:]$ cd /home/chris/repos/toolchain/Archive-Extract/
[canker:(master)]$ perl /home/chris/foo.pl
[20:35:43] /home/chris/repos/toolchain/Archive-Extract/t/01_Archive-Ex
+tract.t .. 
ok 1 - use Archive::Extract;
ok 2 - Archive::Extract->can('types')
ok 3 -    Got a list of types
# 01_Archive-Extract.t
ok 4 -    Object created based on 'tgz'
ok 5 -        No error logged
ok 6 -    Object created based on 'tar'
ok 7 -        No error logged
ok 8 -    Object created based on 'gz'
ok 9 -        No error logged
ok 10 -    Object created based on 'zip'
ok 11 -        No error logged
ok 12 -    Object created based on 'bz2'
ok 13 -        No error logged
ok 14 -    Object created based on 'tbz'
ok 15 -        No error logged
ok 16 -    Object created based on 'Z'
ok 17 -        No error logged
ok 18 -    Object created based on 'lzma'
ok 19 -        No error logged
ok 20 -    Object created based on 'xz'
ok 21 -        No error logged
ok 22 -    Object created based on 'txz'
ok 23 -        No error logged
ok 24 -    No archive created based on '01_Archive-Extract.t'
ok 25 -        Error not captured in class method
ok 26 -        Error captured as warning
ok 27 -            Error is: unknown file type
ok 28 - Archive created
ok 29 -    No errors yet
ok 30 -    Errors retrieved
##### etc etc Ad nauseam until
ok 1706 - Archive type is_txz recognized properly
ok 1707 # skip No binaries or modules to extract x.tar.xz
ok 1708 # skip No binaries or modules to extract x.tar.xz
ok 1709 # skip No binaries or modules to extract x.tar.xz
ok 1710 # skip No binaries or modules to extract x.tar.xz
ok 1711 # skip No binaries or modules to extract x.tar.xz
ok 1712 # skip No binaries or modules to extract x.tar.xz
ok 1713 # skip No binaries or modules to extract x.tar.xz
ok 1714 # skip No binaries or modules to extract x.tar.xz
ok 1715 # skip No binaries or modules to extract x.tar.xz
ok 1716 # skip No binaries or modules to extract x.tar.xz
1..1716
ok     9201 ms
[20:35:44]
All tests successful.
Files=1, Tests=1716, 10 wallclock secs ( 1.21 usr  0.05 sys +  5.56 cu
+sr  3.58 csys = 10.40 CPU)
Result: PASS
[download]

foo.pl is just a path amended version of Khen1950fx's script.

That said, this is different to your original output which appeared to be from being in the perl source tree. Okay, let's have a go at breaking that.

bingos@harlequin:~/repos/perl.git$ cd t/
bingos@harlequin:~/repos/perl.git/t$ ./perl harness ../cpan/Archive-Ex
+tract/t/01_Archive-Extract.t 
../cpan/Archive-Extract/t/01_Archive-Extract.t .. ok      
All tests successful.
Files=1, Tests=1704,  5 wallclock secs ( 0.64 usr  0.02 sys +  2.64 cu
+sr  0.64 csys =  3.94 CPU)
Result: PASS
[download]

I don't know then. The tests in your output appear to be failing before they reach the 'archive' tests, namely because they aren't being run from the correct current working directory.