Note that none of the solutions so far take into account what your original code seems to want to do, which is to only replace newlines before [ATCGN]; OTOH your solution doesn't take into account whether the lines start with > or not. Here is one way that does both, while processing the file line-by-line, thereby saving memory.

use warnings;
use strict;

my $prevline;
while ( my $curline = <DATA> ) {
    next unless defined $prevline;
    if ( $prevline=~/^>/ && $curline=~/^[ATCGN]/ ) {
        $prevline =~ s/\n\z/\t/;
    }
    print $prevline;
} continue { $prevline = $curline }
print $prevline if defined $prevline;

__DATA__
>1
AGTCGTAGCAT
foo
bar
>2
TGAGCTACG
>3
GGCATAGN
quz
>4
CGCACNCAGCTACACC
>5
NGATAGCTACA
[download]

Output:

>1    AGTCGTAGCAT
foo
bar
>2    TGAGCTACG
>3    GGCATAGN
quz
>4    CGCACNCAGCTACACC
>5    NGATAGCTACA
[download]