Well, they say it's never too soon to start a child on the path of learning, but be careful; if he ever goes through a rebellious period in his teens you just might catch him releasing closed-source spyware apps written in Visual Studio. ;-)

I guess if he were my child, and he started to show an interest in computers, I'd start off by making little logic games and puzzles, perhaps, and teaching the kid pattern recognition by playing card games like Concentration or even Go Fish. If you can find a good way to use simple games and fun little puzzles to illustrate concepts like decimal vs base2/base8 (not quite sure how you'd do that) I know those concepts would eventually come in handy for him.

To reiterate what the others here have said, however, I would be very wary of trying to push the young child in any particular direction, and certainly stay away from forcing the issue. If he feels like he has no choice but to do what you say, it might create a psychological barrier when it comes to programming that may be difficult to overcome. I know that I had a deep aversion to playing the piano for many years because my parents forced me into lessons and recitals, which I didn't enjoy at all at the time. Later I discovered a love for the guitar, but it took me a long time to overcome some of my feelings of anxiety and resentment and truly start to enjoy music.

If you are concerned about the visual aspect of programming beckoning him, you might want to look at the older language, Turtle, which allows kids to draw pictures by specifying where they want the turtle to go. It brings back fond memories, and there is probably an open source variant somewhere with updated code.

Above all, congratulations; bringing a life into this world is a precious gift, and you sound like you are going to be an excellent parent.