http://qs321.pair.com?node_id=211998

I had an interesting discussion at work today that I thought I would share as I would appreciate your feedback.

For a particular problem I was attempting to solve, I chose a Hash of Hashes as my data structure. The data structure itself was generated in a subroutine and returned to the calling routine as a hash reference where it was used successfully. Code was not really obfuscated in any way, quite clean and did not resort to rocket science.

I showed the code to a co-worker and they made a rather bizarre statement: "Yep, that's pretty cool but if anyone else has to maintain your code they're screwed". I was rather miffed by this and asked him to elaborate further. Turns out that said co-worker did not understand HoH's or any other "complicated" (his words) data structure for that matter.

So that got me thinking. If we follow my co-workers logic, no code should contain any "grown-up" Perl concept until the whole team is comfortable with that concept. In other words you should "code down" to the least experienced coder on your team. If you don't, the code is perceived as un-maintainable.

So what do you think? Where/when is the line drawn?

When coding, do you assume a certain level of expertise of the lucky so-an-so who has to maintain the code? Do you simplify your code because the "next guy" might not fully understand map or perhaps doesn't yet grasp the concept of references? Do you code-on regardless knowing that there's always perldoc or a copy of The Camel begging to be read if the less experienced members of the team get confused?

Perhaps there's a happy medium - thoughts anyone?

-- vek --

Replies are listed 'Best First'.
Re: Maintainable Code?
by Elian (Parson) on Nov 11, 2002 at 18:04 UTC
    This is a place where coding conventions can come in handy. You (or someone) sets down what techniques and structures are generally acceptable, which aren't, and (by omission) where the grey areas are. Someone should take a look at the tasks generally done, as well as the skill and clue levels of the average programmer in the environment, and set out what's considered appropriate and inappropriate, and where that's true.

    This'll differ from place to place. You may find that Inline::C code is just fine in mainline programs, others may consider it OK in libraries, and still others may decide not to use it at all. The same for fancy data structures, or closures, or threads, or forking servers with SysV IPC and memory sharing.

      Bingo.

      Maintainability doesn't exist in isolation. It comes from interactions between programmer and code. Advanced coding techniques are powerful. Programmers cannot understand everything. The two need to meet in the middle somewhere, and the purpose of coding standards is to agree on where that some should be.

Re: Maintainable Code?
by Callum (Chaplain) on Nov 11, 2002 at 18:12 UTC
    Document everything thoroughly, especially anything that isn't obvious (for some value of 'obvious'), even if it's just with a reference to where they can find more information.
    # hash of hashes -- Advanced Perl Programming, pages 27 - 28 # does foo, because it's better than bar

    You don't need to explain everything, just provide keywords and one line descriptions of what it does. Between the Camel book and Google most things can be found in moments providing people know what they are looking for.

    If you're good enough to be using relatively advanced methods then you're good enough to code and document well, and even if some newbie never has to maintain you code, at somepoint your going to be working on it 2 years after you wrote it at 4am and may want a reminder of what (and why) it is.

    In general I'd advocate not coding down, but allowing plenty of scope for a random maintainer to have to later figure out what you're doing. Coding will provide a definate benefit now, at a possible cost in the future; coding down has a definate cost now, for a possible benefit in the future.

    Code up, use the time saved by coding-up to write more documentation :)

Re: Maintainable Code?
by mirod (Canon) on Nov 11, 2002 at 18:21 UTC

    When the data structures are a little complex (at least for the (not necessarily) intended audience ;--) I pepper the code with calls to Data::Dumper (actually with Data::Denter) that can be activated in debug mode, whether it is by setting a variable, an environment variable or just by uncommenting the print Dumper $HoHoA; statement. This way figuring out what's in the variable is not too hard for the maintainer. I don't usually use them myself actually, I prefer the wonderful x $HoH in the debugger, which works wonders, especially as you can then try print $HoH{foo}->{bar}; and see what comes out.

    In any case not knowing how to deal with a HoH shows quite a limited grasp of the language (and of programing in general as a matter of fact). It might be worth mentioning to your co-worker that knowing how to deal with complex data structures is really required if he wants to be called a programer (you could maybe be a little more diplomatic than that though ;--).

      In any case not knowing how to deal with a HoH shows quite a limited grasp of the language...

      Too right. You can understand why I was so surprised to hear an admission like that. I for one do expect a certain level of expertise from those who would have to maintain my code - especially when the code is not overly complex.

      -- vek --
Re: Maintainable Code?
by Mr. Muskrat (Canon) on Nov 11, 2002 at 18:07 UTC

    If you are using well-documented code, then I see no problem with including "grown-up" Perl concepts. You might want to include a reference in the comments to a book (complete with page number) or website (Perl Monks?) that will explain the "grown-up" Perl concept you are using.

    If you continuously dumb down your code, then how are they ever going to grow up as a Perl programmer?

      In The Mythical Man-Month Brooks said, "Show me your flowcharts and conceal your tables, and I shall continue to be mystified. Show me your tables, and I won't usually need your flowcharts; they'll be obvious."

      I interpret this to mean, "Show me your data and I can understand your code."

      Basically, if you document what your data structure should look like, what the the keys at both levels should be as well as the value of the second level keys, then maintenance should be a snap.

      I use the following notation for hashes (I probably should just use the format that Data::Dumper uses but I like this, too)

      |->key1
      |     |->key1.1 = value1.1
      |
      |->key2
      |     |->key2.1 = value2.1
      |
      |->key3
            |->key3.1 = value3.1
      
      It sort of represents how I've visualized hashes.

        I typically phrase that as "Data Structures are Algorithms" - a pun to Nicolas Wirth fully intended.

        Abigail

Re: Maintainable Code?
by Zaxo (Archbishop) on Nov 11, 2002 at 18:09 UTC

    That sounds like an opportunity for a challenge. Ask him to write it "maintainably". What would you get? Symbolic references? Synchronized arrays?

    After Compline,
    Zaxo

Re: Maintainable Code?
by BrowserUk (Patriarch) on Nov 11, 2002 at 18:58 UTC

    I really hate it when people invoke the "it's a maintenance problem" when the see code they don't understand.

    <obligatory auto analogy>

    Can you imagine (for those of you that didn't have the pleasure), if all cars produced today still had single carbs with manual chokes instead of fuel injection, no electronic ignition, 3-speed gearboxes and no synchromesh, no anti-lock brakes, no air-bags, no self-winders or self-tensioners on the seat-belts, no electric windows, no central locking, no catalytic converters, not to mention all the ICE, self-diagnostics and navigational aids that are beginning to appear.

    All these things add complexity and increase both the need for maintenance and the experience level of those that perform it.

    </obligatory auto analogy>

    Until the production of code and the maintenance of code are given equal footing, this urban myth that dumbing down code saves on maintenance will persist.

    Yes. I know that there are some places one should fear to tread, but using the utility and power of the language to good effect isn't one of them.


    Nah! You're thinking of Simon Templar, originally played (on UKTV) by Roger Moore and later by Ian Ogilvy
Re: Maintainable Code?
by derby (Abbot) on Nov 11, 2002 at 18:36 UTC
    Where's the growth if there's no stretching?

    I say never code down. But at the same time, don't show off - stick with the idioms and ensure your devs know the idioms and know how to use perldoc.

    -derby

    update: This is a bit late, but I don't consider using HoH showing off - they're pretty easy to wrap your head around.

Re: Maintainable Code?
by chromatic (Archbishop) on Nov 11, 2002 at 20:17 UTC

    This is another way pair programming can help. Besides having two heads thinking about the same problem, besides positive peer pressure to write tests and to follow the coding standards, there are two people learning about the code and learning from each other different ways to solve the same problem.

      I love the idea, but does it work when there is so much difference in the level of knowledge?

      That man needs a Perl course before (or gets it during pair programming). I am not sure that this is a situation, when to introduce pair programming.

        It does presume that both developers are reasonably competent. Now you might pair a guru with a newer programmer, but I'd expect that the newer programmer ought to be able to right decent code already.

Re: Maintainable Code?
by rbc (Curate) on Nov 11, 2002 at 18:43 UTC

        Yep. If I had to code-down so that all my coleagues understand I'd have to write everything in Visual Basic. Bleargh

        Jenda

          I think this is why I am suspicious of any programming
          "platform" that comes from MS or has the word Visual
          in its name. When I use these "tools" I get the funny feeling
          that the people that created the tool think they know
          more about what I want the tool to do for me more
          than I do. Or they think I am stupid and they have to
          make everything work so simply that a monkey could do it.

          Of course they make it easy to point and click out Hello
          World demo programs but these tools (I feel) have lost
          the ability to easily represent abstractions needed to
          write more than trivial programs. This is why I love vi and Perl.
    Re: Maintainable Code?
    by dws (Chancellor) on Nov 11, 2002 at 21:27 UTC
      Turns out that said co-worker did not understand HoH's or any other "complicated" (his words) data structure for that matter.

      So your co-worker generalizes from his (lack of) experience, and predicts that everyone else will have a problem. How very human of him. I often catch myself doing the same.

      When coding, do you assume a certain level of expertise of the lucky so-an-so who has to maintain the code?

      I assume that they'll be a notch or two behind my abilities, and try to write code in a way that will help them up the learning curve. This means descriptions of any non-obvious data structures, and no tricks unless well commented.

      Give people good examples to work from, and they may surprise you by how quickly they catch on.

    Re: Maintainable Code?
    by sauoq (Abbot) on Nov 11, 2002 at 23:51 UTC
      "Yep, that's pretty cool but if anyone else has to maintain your code they're screwed."

      Translated: "Yep, that's pretty cool but if I have to maintain your code, I'm screwed."

      That's what your co-worker thinks. If faced with the necessity to maintain your code, he might just surprise himself and rise to the occasion despite his lack of self-confidence.

      In other words you should "code down" to the least experienced coder on your team.

      Absolutely not! You should work together to improve your collective skills rather than work separately to worsen the quality of your code. The "coding down" idea is built on the assumption that each member of the team should be able to do the work of any other without assistance. If that's the case, there is no point in being a team at all. If the least experienced member doesn't understand some code then you should simply help him understand it.

      I think the most effective approach to coding is simply "do your best." Write code that someone at your own skill level would rate highly. Dumbing it down won't help you or anyone in the long run.

      -sauoq
      "My two cents aren't worth a dime.";
      
    Re: Maintainable Code?
    by jepri (Parson) on Nov 12, 2002 at 02:54 UTC
      I haven't flown off the handle for a while, so I'm going to indulge myself here a little. I wouldn't actually say this to someone's face.

      What the hell happened to the time when being ignorant was a cause for embarressment? Since when did being an idiot become something to be proud of? Put in a similar situation as your co-worker, I'd quietly sneak away and try and learn everything about hashes before anyone noticed my lack of knowledge. If I couldn't pull that off, I'd ask for help.

      How did a community with a well deserved reputation for arrogance allow incompetent colleagues to drag us down to their dismal level?

      Forget auto analogies, your co-workers should currently be explaining why they are employed as 'programmers' and not, say, 'street sweepers'. That job would seem to be a better fit for your co-workers, since street sweepers also have no idea about HoHs but a long history of spraying crap over anyone nearby.

      Woo, that felt good.

      In reality, a good way around might be to hold some training sessions so that people could come up to speed with you.

      ____________________
      Jeremy
      I didn't believe in evil until I dated it.

    Re: Maintainable Code?
    by hieronymus (Scribe) on Nov 11, 2002 at 20:01 UTC

      While I do not get near enough time coding Perl as I would like, I have spent enough time reading throug the various Perl books to understand various data structures. Therefore, I would hope it would not be *too* much of a stretch for a full-time coder to comprehend various data structures.

      From a different stand point...as a general rule, if I ever want to get better at anything I do (cycling, coding, etc) I make it a rule to surround myself with people who are better at the particular discipline than I am. So to say that a particular data structure is too "complicated" is in essence saying that he/she does not want to become a better programmer. IMO.

      -hieronymus

    Re: Maintainable Code?
    by jplindstrom (Monsignor) on Nov 11, 2002 at 21:03 UTC
      The only problem with hashes (and hashes of hashes) is that they are seen as "advanced" and noone talks about them until way to late, when the programmer already needs to use... composite data structures. In Perl you (as usual) have to make these for yourself using hashes.

      I remember learning Perl. Until someone mentioned hashes when I asked for data structures, I thought that there was no such thing in Perl! The next, mountain like, hurdle was the syntax for dereferencing complex data structures (and subs returning e.g. hashrefs).

      Explain these concepts by example and you don't have a problem.


      /J

        When I teach a Perl course, I tell them about arrays and hashes on the afternoon of the first day. Even before control structures like if and while (that's the morning of the second day).

        And in the morning of the first day, we talk about the history of Perl, the manual, CPAN, the manual, mailing lists, the manual, websites, use strict, the manual, use warnings, the manual, and how to run a Perl program. Oh, and how to use the manual of course.

        Abigail

    Re: Maintainable Code?
    by sporte01 (Acolyte) on Nov 11, 2002 at 20:17 UTC
      Perfect time to get on the high horse. I have two points of view, but I'll state this much. When writing perl, stick to objects, simple hashes and simple arrays along with references. It prevents you from writing hashes of hashes, which is error-prone. What if you misspell a key? More reason to hate hashes for non-internal reasons (in method/code block) only. I remember once i returned a hash, tried to use one of the keys, it was mispelled, had a false sense of how the code worked, thus creating harder to maintain code as well as buggy. ... but we all have our preferences.

      But anyway, lemme jump to the other horse. It sounds like you guys need a small committee, 2 or 3, to write a coding standards doc. Level the playing field so that everyone knows how to pass data around, how to name functions, etc etc... That's ALL that is needed. So if someone uses a hash of hashes, and it says it is or isn't in the coding doc, you can point it out, and help that person learn that it's ok to use.

        What if you misspell a key?

        My test cases catch it, just as they do if I mistype an array index. :)

          Whilst I completely disagree that the ability to silently typo would ever stop me from using the enormous utility of hashes, there are times when it would be very useful to be able to turn off the autovivification of elements.

          Perhaps this could be added as an attribute in Perl 6 so that it could be controlled on a hash by hash basis? I settle for another strict option so that it could be controlled on a lexical basis.

          Ideally, I think that it would be great to be able to do this lexically on a hash by hash basis. That way you could indicate some parts of your code that you want to autovivify keys of this hash within this scope, where you know you are going to be creating them, but turn it off in other places where you simply want to access the value of existing keys.

          I appreciate that you can use exists to implement this, but its one of those housekeeping chores that I'd really like to be able to have the compiler/interpreter take care of for me. Its less likely to forget than I am:^)


          Nah! You're thinking of Simon Templar, originally played (on UKTV) by Roger Moore and later by Ian Ogilvy
        Mistyping hash keys is something that happens when you use hashes. It's not something that only happens when you use hashes of hashes, nor do hashes of hashes make it worse.

        And you can prevent this from happening by making all your keys constants, and prepending a + in front of each key that otherwise might be autoquoted.

        Disallowing hashes of hashes sounds as useful to me as disallowing arithmetic which involves divisions, as some people don't master division.

        Abigail

        If you accept simple hashes and references, then you have accepted hashes of hashes already.

        As for misspelling problems with hashes, if you use Perl 5.8 then you can lock down hashes without a huge performance penalty, making typos easier to catch.

    Re: Maintainable Code?
    by gjb (Vicar) on Nov 12, 2002 at 00:07 UTC

      My company has two teams, knowledge engineers and software engineers. The knowledge engineers build prototypes in either Lisp or Perl which are eventually transformed into efficient C/C++ code by the software engineers.

      Usually I try to keep in mind that someone has to port my code at some stage, so I tend to avoid writing code that is too Perlish. Of course it happens that I forget and start using Perl idioms that are more difficult to translate.

      Currently one of the software engineers is porting a particular piece of code which is simple in Perl, but non-trivial in C. Realizing this I asked him whether it was not too hard and his answer was pretty nice: "Well, each statement is a bit of a challenge, but understanding it prepares me for the next one, which is just a bit harder."

      Note that our software engineers are very good at their job (and none read PerlMonks).

      Just something from daily experience, -gjb-

    Re: Maintainable Code?
    by PodMaster (Abbot) on Nov 12, 2002 at 16:07 UTC
      Oh boy ... I would've told him <explitive>CENSORED</explitive> ya know (if i am miffed, there ain't no stopping me )
      • Does he know what GLOBS are?
      • Does he know how my/local are different?
      • What are his feelings on until/unless?
      • or the difference between && and and and || and or? (;D)

      Is he an intern or something?

      What did he do to get the job?

      I suggest you make him read perldata, perlref, perldoc -f ref, and then perldsc perlreftut perllol every day until he gets it.

      Once you learn perldata/perlref, there is very little mystery left in complicated datastructures (i feel most anybody with half a brain , and half an hour, can learn all about perldata/perlref with the help of somebody who knows it all ).

      perldata/perlref are as essential as perlsyn (and dare I say, perlre, even perlre is advanced any which way you look at it, but at least knowing how to employ m// and s/// are not).

      update:
      If your dude knows all about arrays/hashes/scalars, point him to References quick reference and let's see if he don't learn it in half an hour.

      ____________________________________________________
      ** The Third rule of perl club is a statement of fact: pod is sexy.

    Re: Maintainable Code?
    by Popcorn Dave (Abbot) on Nov 12, 2002 at 16:45 UTC
      <Rant>

      I think it's time that either your co-worker or perhaps all your co-workers who have to maintain your or anyone else's Perl code check in to their local community college and take a refresher course and LEARN about some of these things.

      When I was learning Perl, HoH's were in the second half of the course I took, but just by reading the camel book I was able to at least figure out how the HoH worked.

      </Rant>

      At what point did people become so afraid of books that they are unwilling to at least try to learn?

      There is no emoticon for what I'm feeling now.

    Re: Maintainable Code?
    by dragonchild (Archbishop) on Oct 16, 2003 at 19:26 UTC
      I write Perl applications. I expect my Perl applications to be maintained by other people who write Perl applications. That assumes a certain level of competency.

      Think of it this way - would you hire a C++ developer who didn't understand templates? function pointers? I didn't think so. So, why on this green earth would you hire a Perl developer who doesn't understand nested data structures?

      If you're a manager and you do that, then you deserve the hell you have consigned yourself to.

      vek - write your Perl applications and be proud of them. Don't pander to the posers who claim Perl expertise, yet cannot understand $x{$y}{$z->('foo')} = $bar->($baz);. You are a Perl developer. Stand tall!

      ------
      We are the carpenters and bricklayers of the Information Age.

      The idea is a little like C++ templates, except not quite so brain-meltingly complicated. -- TheDamian, Exegesis 6

      ... strings and arrays will suffice. As they are easily available as native data types in any sane language, ... - blokhead, speaking on evolutionary algorithms

      Please remember that I'm crufty and crochety. All opinions are purely mine and all code is untested, unless otherwise specified.

    Re: Maintainable Code?
    by kodo (Hermit) on Nov 12, 2002 at 07:58 UTC
      I think using a HoH issn't quite "advanced" really, and shouldn't be a problem to understand even for a newbie. If they don't understand what's going on, they have to learn it, it would be much more obuscating to NOT use a HoH when the situation requires it.

      giant
    Re: Maintainable Code?
    by superpete (Beadle) on Nov 13, 2002 at 08:22 UTC
      I remember the first perl project I was paid for. I needed HoH, LoH, HoL, etc. I read perldsc, and after a few hours of reading and experimenting, I understood it pretty well and got on with my work. Because I was new at it, a few subtle bugs surfaced later during testing, but that happens whenever you learn something new.

      Noone who goes through this continues to think HoH should be avoided because it is "too advanced".

      Try to get your co-worker on a small task where he will be encouraged (forced) by the nature of the task to learn it.