Here's a benchmark I tried that compares sorting to hash-walking for hashes of various sizes, and the results confirm Dominus's expectations. The hash entries are generated using rand(65000) and both subs are given the same hash to work with. (I think the comparison is pretty fair, let me know if you have ideas for either of the subs!)

The code:

sub walkit {
   my $hashref = shift;
   my $min = each %$hashref;
   $min > $_ and $min = $_ for keys %$hashref;
   $min;
}

sub sortit {
   my $hashref = shift;
   ( sort {$a<=>$b} keys %$hashref )[0];
}
[download]

The results:

Size       sort       walk
10      8329.50/s  5289.68/s  (sort wins!)
100     1020.53/s   727.29/s  (sort wins!)
1000      94.68/s    80.14/s  (sort wins!)
10000      7.15/s     7.31/s  (almost a wash...)
100000     0.52/s     0.73/s  (walking wins!)