"This suggests to me (making certain assumptions that the testing population represents the broad user population) -- that only about 4% of Perl users are on Windows."

The basis of your assumption is flawed. Some testers run smokers for several different OS. Windows has licence costs. If you feel Windows is underrepresented it's a voluntary effort, see Why are some OS blank in CPAN Testers Matrix?.

The statistician George Box is often quoted as saying: "All models are flawed, but some models are useful". I'm well aware that the assumption regarding testers being representative of the broader userbase isn't exactly correct. But I'm curious if the estimate is off by a little bit, or a lot.

Based on the types of questions seen on various forums, there could be many more Windows users than the testing logs would suggest. And if that is the case, I'd like to devote some time to testing on Windows.

Just to get you started, a list of dists with failures on the Windows platform in cpantesters within the last month.

#!/usr/bin/perl
use strict;
use warnings;
use feature 'say';
use DateTime;
use Mojo::UserAgent;

# http://api.cpantesters.org/docs/?url=/v3#!/Summary/v3_summary_all 
my $url = 'http://api.cpantesters.org/v3/summary';

# just an example, the last month
my $date = DateTime->now->add( months => -1 )->iso8601.'Z';
my $osname = 'mswin32';
my $grade = 'fail';

my $ua = Mojo::UserAgent->new;
my $res = $ua->get( "$url?osname=$osname&grade=$grade&since=$date" )->
+res->json; 
say "$osname $grade since $date";
foreach my $fail ( @{$res} ){
    say "$fail->{dist} failed on $fail->{perl}";
}
[download]

Displays:

mswin32 fail since 2020-01-19T16:45:05Z
Config-Registry failed on 5.20.1
ExtUtils-MakeMaker failed on 5.22.1
Config-Registry failed on 5.18.2
Config-Registry failed on 5.18.2
CBOR-Free failed on 5.18.2
MySQL-ORM failed on 5.18.2
MySQL-ORM failed on 5.12.2
Metabolomics-Fragment-Annotation failed on 5.18.2
MySQL-ORM failed on 5.20.1
Config-Registry failed on 5.12.2
Class-Refresh failed on 5.18.2
.....
[download]

I don't think there is going to be a method to determine the number of users accurately. Anyone running cpan/cpanm can configure their client to report the installation status (pass/fail) to cpantesters. If you care enough and have the means you could setup an automatic smoker to do this without manual intervention.

Windows has licence costs.

Could useful results be obtained by running Strawberry Perl or Cygwin perl on Wine on a Linux or BSD host? Or perhaps using ReactOS? Or should those be more properly considered different platforms from "proper" cygwin and mswin32?

the Wine comparability layer could IMHO introduce problems. I think this sort of thing would simply muddy the waters.

I would assume the majority are on UNIX variants, but I'm not sure CPAN testers is a reliable source of information for this specific question. For example, how easy is it to set up a windows machine for CPAN testing, as compared with Unix variants?

A semantic question would also be whether you are counting developers or end-users. Maybe it does not matter, though.

And your count also excludes OS's like HP-UX and AIX. I do run CORE smokes on those, but I do not have the capacity to run CPANTESTERS, so you will miss feedback of cpan/cpanm installs on these.

As far as I can tell, none of the automated CPAN smoketest boxes seem to be running Windows, or at least none of the automated testers that will pick up an alpha release. If you want improvements and have a Windows box where you could install multiple versions of Strawberry Perl, you could set up a Windows smoketest box. Be the change you want to see in the world.

It should be possible to run multiple instances using stevieb's berrybrew, and then call berrybrew exec. It might need someone to develop more detailed documentation, though.

I don't think there's a link between the proportion of systems used by CPAN smokers to the proportion of systems used by Perl users in general.

On a different note, the way in which Perl is used might vary by system.

Now if only everyone else running windows but not contributing would reply like this, we'd have some constructive data :p