You didn't run the test with reversed redirection, so the above doesn't actually show anything new. Here's what I get with your code, including also a reversed order test:

~> perl foobar.pl
Normal STDOUT
Going to STDERR
~> echo 'Just a starting point
> > ' >JGBtest.txt
~> cat JGBtest.txt
Just a starting point
>
~> perl foobar.pl >>JGBtest.txt 2>&1
~> cat JGBtest.txt
Just a starting point
>
Going to STDERR
Normal STDOUT
~> perl foobar.pl 2>&1 >>JGBtest.txt
Going to STDERR
~> cat foobar.pl
#!/usr/bin/perl

use warnings;
use strict;

print "Normal STDOUT\n";
warn "Going to STDERR\n";

exit;

__END__

~>
[download]

Which again shows that the redirection order does matter, and in the 2>&1 >>file case, STDERR goes to the output, not the file.

Update: And just to illustrate it in one more way...

~> perl foobar.pl >>JGBtest.txt  2>&1  &
[1] 22571
~> ll /proc/22571/fd
total 0
lrwx------ 1 user group 64 2014-10-29 16:31 0 -> /dev/pts/11
l-wx------ 1 user group 64 2014-10-29 16:31 1 -> /home/user/JGBtest.tx
+t
l-wx------ 1 user group 64 2014-10-29 16:31 2 -> /home/user/JGBtest.tx
+t
~> perl foobar.pl 2>&1 >>JGBtest.txt &
[2] 22578
~> ll /proc/22578/fd
total 0
lrwx------ 1 user group 64 2014-10-29 16:32 0 -> /dev/pts/11
l-wx------ 1 user group 64 2014-10-29 16:32 1 -> /home/user/JGBtest.tx
+t
lrwx------ 1 user group 64 2014-10-29 16:32 2 -> /dev/pts/11
~> cat foobar.pl
#!/usr/bin/perl
sleep(100);
[download]

Okay I stand corrected in that order matters. That said, I honestly can't remember ever seeing 2>&1 not coming at the end of the command line, probably because doing so seems to indicate that it won't be captured to STDOUT as one desires.

~/stuff > cat JGBtest.txt 
Just a starting point

~/stuff > perl foobar.pl 2>&1 >>JGBtest.txt
Going to STDERR
~/stuff > cat JGBtest.txt 
Just a starting point

Normal STDOUT
~/stuff > perl foobar.pl 2>&1 >>JGBtest.txt | grep STD
Going to STDERR
~/stuff > perl foobar.pl 2>&1 | grep STD
Going to STDERR
Normal STDOUT
~/stuff >
[download]

With all of the foregoing comments by both you and me in mind, I think you're first comment implies that it must come first in order to be captured. But in the interest of being complete even if I'm wrong, I included the final tests that does actually demonstrate what is happening on the command line (though it is not intuitive.) Also, it is possible that setting $| = 1; might alter the output ordering.

You must always remember that the primary goal is to drain the swamp even when you are hip-deep in alligators.

Okay I stand corrected in that order matters. That said, I honestly can't remember ever seeing 2>&1 not coming at the end of the command line, probably because doing so seems to indicate that it won't be captured to STDOUT as one desires.

You're entirely correct that wanting to redirect both STDOUT and STDERR is the most common case, and for that one you do indeed need the 2>&1 at the end.

However that does not appear to be what the OP wants; Unless I misread the question, he wants to capture STDERR output using the backticks, while having STDOUT going to the file. For that you need the reverse otder.

With all of the foregoing comments by both you and me in mind, I think you're first comment implies that it must come first in order to be captured.

... in order to be captured by the backticks instead of going to the file with STDOUT.

Also, it is possible that setting $| = 1; might alter the output ordering.

I think you're right about that.