Some brief remarks, taking into account that you're asking for comments on programming style.

I'll refrain from suggesting a module, because that's clearly not what you're asking here, but that might actually be the best answer.

Removing all spaces as in styles 3 to 5 is certainly not a good practice in terms of code readability.

Although the use of index rather than a regex may in some cases be faster, this certainly does not matter in a subroutine that is (presumably and hopefully) called only once in your program. I would recommend using regexes in such a case, as this make the code somewhat simpler.

A Perl beginner would most likely use style 2 ("Neat") rather than style 1 ("Beginner").

Using single-character variables such as $S or $Y for storing the uppercase version of the OS is not the best choice for readability. Using $a is an even poorer choice. A variable name such as $os would already improve significantly legibility.

In style 1, you don't need parentheses for the postfix conditional. Removing them would tend to reduce line noise.

As you mentioned yourself, the subroutine may not work properly (as an example, you'll end up with a Windows separator if you're using Cygwin), but, with this limitation in mind, this is a suggested variation on style 1:

sub get_path_separator {
    for ($^O) {    # stores the OS in $_ to simplify the regex syntax
        return ':'  if /mac/i;
        return '/'  if /cyg/i;
        return '\\' if /win/i;
        return '.'  if /vms/i;
        return '/';
    }
}
[download]

My other favorite would be style 3, but spread over several lines to make the intent clearer:

$_ = $^O;
my $separator =  /mac/i ? ':'  :
                 /cyg/i ? '/'  :
                 /win/i ? '\\' :
                 '/';
[download]

Please also note that launching Perl from a bash terminal under Windows, you might get something like "msys" as the value for $^O (I don't know why).