In my experience I have found that, as in many disciplines, there are those who were born to it & those who are made. The very best are usually those who are both. Like a Mozart type, with an inherent gift, and a Maestro for a father to encourage him. (..or make him into a trained monkey...)

For example, I worked with one gentleman once, a Russian immigrant, who spoke broken english. In 6 months he went from knowing only HTML & minimal JavaScript to becoming one of the best Java programmers I've ever met. This guy was in his 40's when he did this. That's natural talent. (Plus the inability to BS w/coworkers because of the language barrier meant no distractions..)

But then there's the rest of us. I started programming when I was 6 & I'm 30 now. I'm not Merlyn either or Vroom for that matter. But in my own circle of influence I make a difference. How do I get better? By wanting to learn. That's what makes all the difference. It makes a difference in who I hire also. If you want to learn then I'll hire you in a minute over someone who's been writing nothing but CGI's for 3 years & doesn't know how to use mod_perl for example...

So what does that mean? That means that you're going to act a little geeky, in that you might spend your lunch hour reading tech books, like all of the O'reilly books that are pushed by everyone here with a semi-religious zeal.. (Merlyn helped write a few after all..) or kill time in the office by decyphering how your favorite module's code works.

It also means that you DO learn the 'wierd' syntax. Learn the code that you don't know & without question you'll find uses for it. And then you WILL use it. Then you'll open your eyes a little more & realize that you weren't the first person to figure that out & you start learning new ways to do things. After a while you'll start realizing that all of these 'gurus' you worship are just plain folks like yerself. They just spend their time differently. They learn. What's more, they're not these inapproachable demi-gods, they're actually interesting people to talk to. & yes you'd be a little snippy too if you kept answering the same simple FAQ-entry questions all the time. But have you ever stopped & realized that when people here take the time to post code, they've taken the time to write about it, think about it, agonize over it, etc... Especially when it's not their code, but rather a reply, you come to understand that these people really (say it with me now) want to learn.

I know for me, sometimes I feel like I'm searching for the Holy Grail. The truth is you'll never be Good Enough. When you are, that means you're done learning & then well, it's time to die("Say goodbye to the nice people\n");

After a while you'll start realizing that all of these 'gurus' you worship are just plain folks like yerself. They just spend their time differently. They learn.

As an example, I can't tell you a thing about competitive professional sports or current sitcoms. I don't watch TV. I don't follow games. People have to explain to me when I'm watching the SuperBowl with them (to be social) that something that just happened was "good" or "bad". And I haven't seen Ally McBeal, and didn't watch my first Seinfeld until prodded by others, and after it was already in syndication rerun hell.

What I spend most of my spare time doing is being online. Researching the next item, reading about technologies, being here and on Usenet and IRC, answering questions. Well, and the twice-a-week trip to either the Karaoke bar or the live comedy club. And on airplanes, I'm just as likely to read the latest book from O'Reilly as I am a good SF novel.

Now project that backward 30 years. I didn't do sports in high school (I was the runt that got picked last anyway). I spent much of my time in the library, or at home reading whatever books I could get my hands on.

So, to get here, I traded all that "normal" stuff. I gave up my childhood for my career.

Would I do it again? Probably. But don't ask me any questions about our Portland Trailblazers; all I know is the stadium creates traffic jams occasionally, and I can't even tell you what part of the year the season is.

-- Randal L. Schwartz, Perl hacker

Says merlyn:

As an example, I can't tell you a thing about competitive professional sports or current sitcoms. I don't watch TV...What I spend most of my spare time doing is being online.

I suspect that this is a red herring. You waste a lot of time on IRC. If you stopped doing IRC you would watch TV and get just as much done.

(This isn't a suggestion, of course.)

I don't think anyone can be 'productive' all the time. But some people watch TV in their downtime and other people watch IRC; I don't think it's really that different.

--
Mark Dominus
Perl Paraphernalia

No TV. No sports. You just went up a notch in my book! :)

And not all of us consider that such things "normal." On the contrary, those things keep the masses amused and distracted from doing "normal" things like learning, exploring, and asking difficult questions of themselves and their surroundings. Why have people partake in debates, communicate with their congresspeople, and learn philosophy and science when they can vegetate and consume instead?

How many movies do you watch? :-P

The two main ways I get better at programming are (1) doing it and (2) reading about it. Programming is absolutely one of those things where you need the practical experience of making mistakes, discovering stuff for yourself and trying out new ideas, that you will never get if you just read books.

However, reading is almost as important to pick up skills. I read many books and just try to absorb as many ideas as possible. I don't usually worry about the details -- I just try to remember that something is possible, that that there might be an alternate solution, that this data structure might work, that there might be a module for this type thing... Then when I come to coding and I run into a given problem I usually remember where to turn. Try reading books like "Programming Pearls" & "The Practice of Programming" which teach you about the craft. Also a lot of people recommend "The Pragmatic Programmer" although that's not a favorite of mine.

Another thing I'd recommend is to have a very good knowledge of all the standard commands. I know that you don't *need* to know them to get a lot done but it's time-wasting and error-prone to roll your own functionality. I can't count the number of times I've seen programmers at work rewrite existing language functions just because they weren't aware of language features.

One way I reassure myself that I'm always improving as a programmer is by reading my old code, from even as recently as 6 months ago. I almost always feel that I would write it better if I did it again.

It can be hard to tell if you're improving, because the process is gradual... but I bet you have improved a lot over the last 6 months. Just keep coding and reading :)

Says dze27:

Try reading books like "Programming Pearls" & "The Practice of Programming" which teach you about the craft. Also a lot of people recommend "The Pragmatic Programmer" although that's not a favorite of mine.

Warning: This is going to be a weird article. I would probably not post it at all, except the original poster cited me as an example. This is what has worked for me; it may not work for people with very different mentalities.

If you want to become a strong programmer, amaze your friends, and confound your enemies, one thing you should probably do is try to read a lot of books that are not what other people are reading. We have a community in which all the members read the same books. Then everyone knows the same things, and discussions of technique turn into a lot of navel-gazing.

I recommend: Find two new languages that nobody else you know knows how to use. Then do a couple of projects in each language. Take notes. As you read the language manual, judge every sentence. Ask yourself "Why was it designed this way? Is this like other languages? How are other languages different? Is it better or worse? Why? What would this be good for? When would I want to use this?" and so on. Take notes. Yes. Take notes. Then when you do the projects, revisit the notes. Notice what you were right about and what you were wrong about. Write down the stuff that you didn't appreciate at first and the problems that didn't appear until later.

This is not like in college where you write down a whole bunch of boring crap that will be on the test. Here, the goal is to write down only the interesting stuff that occurs to you while you are reading. If no interesting stuff is occurring to you, there is a problem. One problem is that you don't find the material interesting. In that case you should trade it in for something you do find interesting. A more likely problem is that you may not know how to read. This is not a joke---most people don't know how to read. They think that the meaning will magically jump from the page into their minds. But it doesn't. You have to read actively. Read once sentence, then think about it for a long time, and ask yourself the questions I listed above. Then read another sentence. You are going for depth here, not quantity, because the goal is not to learn language X, but to become smarter.

Writing stuff down makes you smarter. It gives you an infinite memory. Smart people can sustain a train of thought for maybe fifteen or twenty minutes if they are not disturbed. If you take notes, you can carry on a sustained train of thought for months. It's amazing how few people do this.

Do the same thing when you read the Perl manual. Read a sentence. I will pick a sentence from the Perl manual at random: Line 2000 of perlfunc, whatever that is:

getpgrp
Returns the current process group for the specified PID.

Great example. What is a process group? What is it for? When would I want to use getpgrp? What is an actual application in which I would need getpgrp? What is another? What will getpgrp actually return? An integer, or what? What would be the best design here? (Then if the next sentence reveals that they didn't use the best design, try to figure out which one of you screwed up.) Can this call ever fail? When? How? How will it report the error? What might the return value be in that case? (Then read on to see if you are correct.) Are there permission or security problems here? Would it be dangerous to be able to get the process group ID for someone else's process? And so on.

Then you go on to the next sentence. If you read the whole manual like this, you will progress. That's what you wanted to know, right? How to progress.

Another thing I suggest: Find out what books everyone else is reading, and then read different books. Then you'll be well-placed to solve the kinds of problems that the people areound you haven't been able to solve. What if everyone owned a hammer and nobody owned a screwdriver? If you know things other people don't, the community will be stronger. Here are some books I liked that other people do not reommend: Paradigms of Artificial Intelligence Programming, by Norvig; Computer Networks, by Tanenbaum; Principles of Program Design by Michael A. Jackson (which is a brilliant, brilliant book that nobody reads); Software Tools by Kernighan and Plaugher. It's tempting for me to suggest a bunch of other random crap that I found enlightening, but that would be missing the point---you need to follow your heart, find the stuff that you find interesting, and read that. If you discover yourself saying one day "Gee, I keep hearing people talk about CORBA, but I don't know what it is," then get a book about CORBA and find out. And take notes.

The books I like are the ones that make value judgements. You should not be afraid to make judgements. Engineering depends on judgements. There are usually four or five ways to do something, and most programmers pick the first way they think of, which is why software is so crappy. To do good engineering, you need to pick the best way. It doesn't do any good to get all mealy-mouthed and say "Oh, but which way is 'best' will depend on circumstances, so you can never really know, wah wah wah." It's the engineer's job to know. You are in the circumstances, you know the circumstances, and you have to make the best decision for the circumstances. That's what engineers do. Reinforced concrete, or prestressed? (This week I'm reading Why Buildings Stand Up by Mario Salvadori.)

Also I would agree with Randal's point that most people waste a lot of time doing stuff like watching the TV. It's really easy to waste a lot of time watching a lot of crap on TV. TV-watching is often a waste of time because it is so easy to do it passively; you just sit there and let the pictures go by. I like to watch TV with my wife, and while the show is going on we talk about what has happened, what might be coming up, and why the characters are behaving the way they are. Of course, this requires that you have the right wife; I can't offer any useful advice about that. :)

People waste a lot of time with novels in the same way that they do with TV: They just turn over the pages and let the words flow into their minds without thinking about what they are reading---so don't. Read actively: "What is James feeling? Why did he do what he just did? What will Lydia's reaction be? What will the outcome be?" I like to read aloud sometimes, because actually hearing the words is different from reading them. Your brain isn't built to read language; it's built to hear it. When you actually hear the dialogue and description in a book, all sorts of meaning becomes much clearer. Of course, if you do this, you're likely to discover you've been reading a lot of crap. ("Gosh, nobody talks like that! Why didn't I realize before how silly this sounds?") Oh well!

To summarize: (1) read and do things that other people around you are not reading and doing. (2) Read actively, asking questions and taking notes.

This is what has worked for me. I hope it is not too peculiar.

--
Mark Dominus
Perl Paraphernalia

    Although it stands to reason that a samurai should be
    mindful of the Way of the Samurai, it would seem that we
    are all negligent. Consequently, if someone were to ask,
    "What is the true meaning of the Way of the Samurai?"
    the person who would be able to answer promptly is rare.
    This is because it has not been established in one's mind
    beforehand. From this, one's unmindfulness of the Way
    can be known.
[download]

--

As far as telling how fast and how far are you progressing with a tool, or in this case a programming language, all I can share is my own method.

Take something simple you wrote along time ago, (in the world of computers this would be 8-3 months ago :) ) now take what you wrote and write the same thing but as you would do it now, disregard any of your old code and just do it as though it is a problem and a task presented to you at the present moment. Now take those two peices of code and evaluate the quality of both, the chances are that if you have learned anything at all since you first wrote said peice of code that the newer one is going to be faster cleaner and what not.

Another way one can look at is this. Take a problem from this site, maybe something that is written about in Seekers of Perl Wisdom and that is replied to with code as a solution by a perl programmer you respect and beleave is better than you, aka "a good perl programmer". And without looking at the code they posted, write your own solution and then compare there code vs. yours.

An example of this little method of mine that I ran into once was a situation where I wrote a peice of code for one of my clients. It was supposed to be a simple image manipulation script, that would take images in a directory and process them in a way and then store them somewhere else. Now mind you this was about a year ago and I knew a whole lot less about perl than I do now. So I wrote this script and it was about 200 lines of code and was really ugly. Well recently that same client asked me for some more things (features) to be added into that script. Well once I started on adding those features into the script and took a look at the code for the first time in about a year, I was amazed at how horrible the code was. Well I rewrote it from scratch, and now its about 50 lines of much cleaner code and 800 precent faster with new features and all (where as the new script processes the same image in 1/8th the time the old one would).

I guess what it really comes down to is whether or not you are happy with how good your code is. And that all depends on how you view your code in comparison to other people code and that all depends on how good you view there code.

Well anyways that just my two cents, and Iam going to stop babbling now :)

/****************************/
  jason@gost.net, wh@ckz.org
    http://jason.gost.net
/*****************************/
[download]

Like merlyn I don't have a lot of the usual time-wasters.

Like Dominus I like being engaged in whatever I am doing.

Unlike both I don't have decades of programming experience. It still is under 4 years, and I have a lot of gaps. (That I am trying to fill in as I go, but by the time I hit some of the big ones, I am sure that progress will create others.)

Now for my weird advice, read Re (tilly) 1: Discipline, then RE (tilly) 3: On elegant coding... and you should get a good insight into what my basic learning strategy is. I have direct evidence that the first works for other people. I have no evidence that the second is how others work. I know that neither is something that most people would consider trying.

And finally a controversial question. Talent or effort? I don't know. I have seen plenty of stories of people who demonstrate that it is supposedly talent. I am positive that there are plenty of people who would add me as yet another example. Yet after I look around from reading yet another book, considering another opinion, writing more code, it sure feels like effort. And sure I have only been programming a short time. But I started with years of experience in math. I didn't have to learn the basics of efficiency, recursion, breaking problems up or paying attention to detail. I started programming with those skills more highly developed than most veterans of programming.

If I thought that anything real was known about nature vs nurture and talent, I would probably go out and do some research on this question. But the fragments that I saw the last time I got curious did not inspire confidence. Perhaps after they have a while to analyze the human genome, I will take a peek to see what the orthodoxy turns out to be...

Reading the Ruby book was *very* enlightening, because I'd come across a Ruby feature and see how that would work in Perl... sometimes I could even see how to implement it.

For me, what works is a combination of natural curiousity, some talent, and a willingness to play with things until they make sense. There's probably also a part of my brain that sees patterns wherever I look.

I was reviewing a book about Unix system administration and came across a chapter on UUCP. It's not as widely used these days, but just reading about how it was implemented gave me ideas on how to do something completely different. Reading some documentation on Apache functions available to modules gave me other ideas.

The important thing is being able to look at your problem in a new way, and to be able to learn from lots of different places.

If you have *some* knowledge and experience, and the will to do something about it, all you have to do is capture those creative flashes when everything turns sideways in your brain and you Get It, for just one moment. That's all.

I think i am coming to understand that one of the many good ways to progress in your skills is to be part of the community (which of course you are doing by being here). By keeping tabs on what problems people are solving and how they are solving them you'll encounter issues that may not have arised yet in your particular employment/hobby focus.

When I decided to register here i thought it would be "fair" if I tried to answer a question (one i could answer intelligently and legitimately, of course) for every question I asked. I've quickly found that answering other people's questions when i can is quite rewarding, and it makes me even more interested to see what others who also respond to the question have come up with. It is especially interesting to compare one's own response to solutions provided by those "gurus" among us. It gives you the chance to understand the difference of approach you may have. (note I am not saying that difference of approach is bad. I am saying that understanding multiple approaches to a solution is a very good thing).

I've found that one of the best ways to learn something is to talk to people who know it, or listen in on them. I wanted to get into Perl internals, so I subscribed to perl5-porters. At first I didn't really understand what people were talking about, but over time I picked up bits and pieces. Although I still haven't touched the code, I have a better idea than most of my peers how Perl works on the inside, and I feel confident that I could probably do some work on it.

Last week, I posted a message on perl6-internals suggesting an alternative way to store my() variables. (In Perl 5 they store them as an array of arrays, one nested array for each level of recursion; I thought that we should use a symbol table like we do with globals, local()ing things when we recurse.) Once people understood what I was really suggesting (some people had trouble decoupling the storage of globals from the semantics of globals) they came up with reasonable arguments against it. (Basically that it would take up more space, overuse the savestack (the data structure that holds the old values of local()ed variables), require more startup time if we were reading in bytecode from a file, and might be slower. The advantage was that we wouldn't have to write or debug as much code in the core and that certain lookups which must happen at runtime would be quicker.) When I pointed out an optimization they could do, they nodded in agreement (so to speak) and pointed out that it would still be bigger in memory; however, Dan Sugalski (the guy in charge of internals for Perl 6) conceded that there could be significant advantages to my approach and said that they might benchmark the two approaches and compare them.

I know that, had I not been keeping an eye on perl5-porters, I would have had no idea about any of that. I also know that, if someone else had posted that message, I wouldn't have understood it. But by paying attention to other programmers I learned a lot about a subject I cared about.

---

On a separate note, you mentioned about taking days to come up with the plans for things. That's not a problem at all! My father is a programmer (he learned on a PDP-11 at UCI). He's been doing this stuff for almost thirty years, and he still takes days just writing down how he thinks a new program should work. His programs are generally well thought out and fairly quick and understandable--even to me, someone who doesn't even know that language.

Some of the best coders take time to think up how to do something. Don't sweat it--if you really are good, you'll come up with the right technique eventually.

Update: Lo and behold, I've been made Perl 6's first Configure pumpking. See, listening works!

=cut
--Brent Dax
There is no sig.

Mothera and anyone else ...

I am currently still in Universtiy taking Computer Systems Enigeering. Every day I wake up with plenty of new tasks on my plate and a new, totally forgein, problems to try and solve. Often it is very tough to stay afloat ... let alone try to swim with the sharks !!

One puts in horrendus amounts of time sometimes and still no concrete solution is reached. Sometimes, one puts in 5 minutes and the problem is solved completly. If you ask a peer they may have done it in 5 minutes compared to your 20 hours ... or vise versa. This gets very frustrating and sometimes you feel like you are making no progres and you even feel pretty stupid sometimes.

This was very hard for me to cope with at first ... but I have been kinda forced to learn that each person is VERY different from another. I basically comes down to YOU. It is all about genetics and drive. Some people will learn faster, some people will push themselves to learn more, some people will burn out trying to do more, some people will never really know ... and some people will give up at the first sign of trouble. This applies to ALL ranges of intellegence and smarts (I consider the formentioned to be very different). YOU make the choice and YOU have to live with whatever choice YOU make. (YOU meaning anybody who is reading this)

After many hours of worrying, thinking and everything in between I have come to this simple conclusion:

Keep fighting for what you want .. it will come.

I work-out and run to de-stress ... and one day while running I had a thought that seems relevent ... at one point in time I couldn't crawl... then I learned to do that ... then I learned to walk ... then run ... now I train myself at a much higher level. (I was running my daily 5km) It often seemed that I had reached a limit ... but then one day further down the road I realized that I had passed it a long time ago.

In this context, I just had a full year of Java and Assembly programming courses at school. When I started I knew nothing of either. I was excited to learn. As the courses progessed I ran into problems of varying degrees. Some I solved all by myself, some I had help on and some I never solved completely (argh). Exmas ended last Thusday. I may not be a pro Java, C++, ASM, Perl etc. programmer ... but I know more than what I knew before ... and I am trying new things daily. Sometimes I feel stupid and other times I feel like a God ... it all depends on what is going on. The most important thing is that I have not ... and will not give up. There will always be people who are better at this than me ... but as long as I am pusing my limits I am happy (in retrospect or after some thought).

With respect to your situation ... I think that you just need to chill ... and keep plugging away at things. Like everything ... in the end you will get out exactly what you put into it. One day you will look back and say ... "I posted that, wow, I really came a long way !!"

There will always be a new challenge and a new plateau to reach ... and then surpass. Keep climbing and keep fighting !!!

Ducati

============================================

"We rock the body to rock the party ... until the party rocks the body"

De La Soul

How long did it take you before you felt like you really started to get it?

It would also seem though, that just books about being a programmer aren't necessarily what you're looking for, although like i said, i could be backwards. i would suggest books like Mastering Algorithms with Perl, or maybe an AI book, or maybe a C++ data structures and algorithms book. Something that goes into the theory and thought behind the programming that will allow you to see with more insight how you want to lay out a program because you have more knowledge of ways to lay out a program well, in any language. Most data structures are possible to create in any language, and sometimes it's far better to borrow from another language than stay true to the language.

Not a particularly good example, but for an AI class i had to make a square tile floor that related position, whether it was dirty or clean, and other such features, in Java. What i did was create a perl-esque hash, so that with 4 methods and two arrays i could keep track of all of the properties of any one tile easily. Everyone else had about a dozen or two methods and had to do interesting things to keep everything happy. It wasn't necessarily the best solution, but i think it was the most maintainable and flexible, because if i changed two lines the tile had more properties, everyone else had to write two (or more) functions as well...

It was because i've studies Perl and knew hashes that i thought of that. It's because of my (little) knowledge of C++ sorting and searching that i can write many of the programs i do. If you learn new techniques they'll stay with you no matter what language you use, if you jsut learn the language or how to be a political programmer, you'll be stuck in one type of job or just be some suit behind a desk...

Hope That Helps,
jynx