This is btw another proof why this is not the real color/number correspondence: that's only the deafult palette, you can change it to whatever you like

VGA cards use it as the default for backward compatibility. It was introduced by CGA cards, and on those it was not merely the default and could not be changed; it was hardwired. (CGA cards did not have any way to display a color given arbitrary red/green/blue values. They *only* had the sixteen colors, ever, period. As an added bonus, you could only get more than four of them on the screen at once in text mode (although there were tricks available to make text mode appear to be a low-res graphics mode).)

I was pretty sure the CGA colors also match the ANSI colors, but I shouldn't be surprised that the VT100 used a different scheme; DEC did a lot of things differently from the rest of the world, such as using Category 4 (not 3, not 5) cable and MMJ connectors for networking, or the extraordinarily wacky way VMS expresses directory paths.

It was introduced by CGA cards, and on those it was not merely the default and could not be changed; it was hardwired.

Yes, you are right in this one. I do not know which of these numbering schemes was the first, any of them might predate ansi or vt100 terminals or cga cards as far as I know. (ZX spectrum seems to use 0=black, 1=blue, 2=red, 3=magenta, 4=green if this emulator is right.)

As an added bonus, you could only get more than four of them on the screen at once in text mode (although there were tricks available to make text mode appear to be a low-res graphics mode).

(Update: I was wrong here, see jonadab's reply) I think this is wrong. There is a low-resolution color mode of size 160x100 pixels (lines are doubled), 16 colors. That mode, however, is not supported by the bios, and the resoultion is very small (that's easy to say now, sitting at an 1280x1024 tft screen, mind you), so I guess only few programs used it.

Update 2015-12-01: see also https://en.wikipedia.org/wiki/List_of_color_palettes.

There is a low-resolution color mode of size 160x100 pixels (lines are doubled), 16 colors.

That's one of the two available tricks to make text mode look like a low-res graphics mode. The other, easier to understand trick is the 80x50 mode that the Nibbles game (example code distributed with QBasic) used. The 160x100 mode is more difficult to achieve. The 160 part is easy to understand: it's 80 doubled, using IBM Extended ASCII block-shaped characters to split the text character cell vertically, in pretty much exactly the same way as Nibbles splits them horizontally. Getting 100 characters high instead of 25 is the hard part; apparently there's a little-known byte you can tweak someplace to change the character cell height. This is of little use normally, since with a height less than about 7 you could never read the text, so it's seldom mentioned or discussed, but there you go. (Of course, you already knew there was enough video RAM on a CGI card to hold 200 lines' worth of characters for text mode; this capability was usually used for quickly switching between several available screens, e.g., for help screens, "boss" screens, and the like.)

But if you use a "real" graphics mode (e.g., the 320x200 "medium resolution" mode), you only get four colors max, and three of them cannot be chosen independently; there are three available hardwired palettes. The fourth color is the "background" color and can be any of the 16 colors.