http://qs321.pair.com?node_id=820672

Chapman: One of the cross beams has gone out askew on the treadle.
Cleveland: Well what on earth does that mean?
Chapman: I don't know -- Mr Wentworth just told me to come in here and say that there was trouble at the mill, that's all -- I didn't expect a kind of Spanish Inquisition.
Ximinez: NOBODY expects the Spanish Inquisition!

-- The Spanish Inquisition by Monty Python

As the Monty Python boys might say: Nobody expects the Agile Imposition! At least, I didn't. Though I noticed some of the cross beams of our development process had gone out askew on the treadle, I only expected some minor adjustments to our existing process -- not an agile imposition.

I was also taken aback at the way it was introduced. With little prior consultation or discussion, all development teams were instructed by management to follow the Scrum development process, effective immediately.

Imposing a process on a team is completely opposed to the principles of agile software, and has been since its inception... An important consequence of these values and principles is that a team should choose its own process -- one that suits the people and context in which they work.

-- Martin Fowler

As for the rationale for this Scrum imposition, development managers are typically under intense pressure to improve productivity. Accordingly, our management team is always on the lookout for process improvements. Scrum was successfully tried on a pilot project, so there was a natural urge to apply it more broadly.

Please remember that just because a technique worked for you last year and for one project, it does not follow that it will work unmodified for someone else or for a different project.

-- Bjarne Stroustrup, The C++ Programming Language, Chapter 23, Development and Design

Flushed with the initial success of the pilot, there may have been some unrealistic expectations that this new process might finally deliver the long sought after order-of-magnitude productivity boost. Yet software development remains hard. There is no silver bullet.

What disturbed me most about this little episode was there seemed to be no place for common sense in this well-intentioned attempt to improve the way things are done. When I suggested team-specific adjustments to their process, it was implied that I "didn't get it" and that I'd benefit from attending another Scrum training course, maybe one day even attaining the coveted "Certified Scrum Grand Panjandrum" title. I felt sad and alone, a lowly novice in this new religious order. Most developers, and all managers, seemed to disagree with my points of view.

For therapy, and as a reality check, I'm writing this new series of articles. I encourage you to tell me where my cross beams have gone askew on the treadle. Your feedback, anecdotes, opinions, advocism, and any citations you may know of, are welcome.

Software Development Methodology "Science"

This type of experimental design seems to be missing from current empirical software development research. Many will argue that such experiments are impossible, or unreasonably expensive, when studying how highly paid people perform lengthy complex tasks. That may be true, but it doesn't justify misrepresenting anecdotes as scientific results.

-- David Parnas in IEEE Software Magazine, Nov/Dec 2009 (page 58)

I'm generally skeptical of software development methodology "science". Stronger, I'm profoundly skeptical of claims like "we measured a 400% productivity improvement by switching to the amazing new XYZ development methodology!!". Apart from the obvious conflict of interest in measuring the productivity of a methodology you're making money from, performing anything resembling a valid scientific experiment comparing development methodologies is dauntingly difficult and often prohibitively expensive. After all, how on earth do you "prove" that a methodology that "worked" for one team on one project will similarly "work" for a different team on a different project? And how can you "prove" that the original team would have performed the said project better (or worse) had they adopted a different methodology? Lacking a time machine, you can't just re-run the experiment. The best you can hope for are fuzzy averages and statistics for your organisation. Which may or may not apply to different organisations. It's a very difficult problem.

Should Teams be Allowed to Choose Their Own Process?

It turns out that there is no one way to motivate people, no best way to organize teams, and no universal rule governing interactions. In addition to cultural differences, each individual has unique strengths, styles of learning, and values.

-- Leading Lean Software Development by Mary and Tom Poppendieck (p.187)

Each project's ecosystem is unique. In principle, it should be impossible to say anything concrete and substantive about all teams' ecosystems. It is. Only the people on the team can deduce and decide what will work in that particular environment and tune the environment to support them.

-- Communicating, cooperating teams by Alistair Cockburn

It's never a good idea to force someone to practice XP against his will.

-- The Art of Agile Development by James Shore & Shane Warden (page 47)

I'm interested to learn:

Notice that competent individuals, and teams, are refining and improving their processes all the time. I contend that it's better for process improvements to be driven by individuals and teams -- rather than being mandated from above -- because this improves morale and motivation (via empowerment) and makes it clear that improving process is everybody's job and that it needs to be performed constantly and forever. For what it's worth, I understand that both Google and Microsoft allow each development team to choose their own process.

People versus Process

Generally, I feel it's better to focus more on people than process. What do you think?

The closest I could find to "scientific" support for my point of view is:

No shortage of anecdotes though:

Given the importance of motivation, I'm interested to learn what motivates you, what are good ways to motivate and reward programmers.

Individuality

It's the team that matters. Where would The Beatles be without Ringo? If John got Yoko to play drums the history of music would be completely different.

-- David Brent

Another point I'm interested to hear your opinion on is the popular management notion that programmers are "equal and interchangeable". While making projects easier to "manage", I feel this fallacy harms the motivation of programmers who take great pride in their work. At least, I gain much more joy from becoming an expert than from performing an "average interchangeable" job with little regard for quality or craft.

Managing Change

And it should be considered that nothing is more difficult to handle, more doubtful of success, nor more dangerous to manage, than to put oneself at the head of introducing new orders.

-- Niccolo Machiavelli, The Prince (1513)

The fundamental response to change is not logical, but emotional.

-- from Peopleware, Chapter 30, Making Change Possible

This is much harder than it looks. Many people, especially older folks like me, naturally resist being changed. Transforming a productive and proficient oldbie into an ineffective newbie doesn't seem like a good idea to me. It's vital to explain to people why they should want to change, and so cultivate readiness, not resistance.

Process Religion

Scrum as an institutional change agent is invaluable to a church.

The law in Scrum that nothing new can interrupt a sprint plays havoc with the life of a minister ... unanticipated short-term emergencies happen every day! People die. Pastoral emergencies emerge. These cannot be scheduled.

Vocabulary can be a stumbling block. Many church people associate words such as "product owner" and "value added" as tools of a business or corporate environment that are at best suspect. "We are a church," they protest, "not Microsoft or a bank!" I have experimented with "vision holder" to describe the product owner.

-- from Scrum in Church co-authored by Scrum co-founder Jeff Sutherland

The final section of this first episode notes a lamentable aspect of human nature: well-intentioned process zealots who get way too excited about their methodology. Here's an example taken from the book Agile Project Management with Scrum by Scrum co-founder Ken Schwaber.

The Sprint felt dead. As ScrumMaster, I could have called for an abnormal termination of Sprint. Circumstances had changed so that the Sprint goal appeared to be unobtainable. I just couldn't do this, though. WebNewSite had just started with Scrum; I had told them that the ScrumMaster removed impediments. This was certainly an impediment -- Thomas couldn't be reached. The more I thought about it, the more I felt that this was unacceptable. Even though Thomas hadn't left a phone number, I was sure that he wouldn't want to leave the team in the lurch and that he would want to immediately come to the team's aid if he only knew of its predicament. But how could I contact him?

Unfortunately for Thomas and his vacation plans, I'm a fan of mystery novels. "Of course!" I thought. I'll hire a private detective to find Thomas. After some searching, I found an ex-FBI agent who had an office in Billings, Montana. He was excited about working for an Internet startup. He found Thomas within two days. Thomas was able to assist the team, and the Sprint goal was met. I figured that I had been pretty inventive, spending only several hundred dollars to get around a major impediment in an unconventional manner.

-- from chapter eight of Agile Project Management with Scrum by Scrum co-founder Ken Schwaber

Though Ken seemed proud of his unorthodox antics, divulging an employee's social security number to a private detective so that he can locate him and drag him back from vacation to work on a Scrum sprint seems like a violation of common sense to me (and may even be a violation of common law). Certainly, I would feel violated were this done to me. What do you think?

It is easy for an individual or an organization to get excited about "doing things right"... common sense can be the first victim of a genuine and often ardent desire to improve the way things are done. Unfortunately, once common sense is missing there is no limit to the damage that can unwittingly be done.

-- Bjarne Stroustrup

Other Articles in This Series

References

References Added Later

Updated 3-feb: Minor improvements to wording. Added finishing Stroustrup quote.

  • Comment on Nobody Expects the Agile Imposition (Part I): Meta Process

Replies are listed 'Best First'.
Re: Nobody Expects the Agile Imposition (Part I): Meta Process
by ruzam (Curate) on Feb 02, 2010 at 00:37 UTC

    Sigh... I've been in IT long enough to see so many methodologies come and go that I've long since given up caring about any of them.

    The methodology is the end product of a work culture which is an end product of the business itself. You can no sooner transplant a development methodology from one business to another anymore than you can transplant a monastery's methodology for making cheese for an life insurance company's methodology for selling policies.

    I can recount one project in a business I once worked for. According to an IBM study of successful startups a 'Rapid Project Development' methodology was created. Some of it's key elements were tight working conditions (elbow to elbow desks), drop dead deadlines (deadlines are never pushed back, ever, for any reason) and work 'till it's done schedules (overtime, overtime, overtime). Now when you're fighting with your savings on the line to bring a business you're passionate about to life with your like minded buddies, you'd do all of these things and then some without giving it a second thought. Not the least of which the rewards would be wilder than your dreams could imagine. Now this same 'highly successful' methodology was transplanted into a more than 80 year old, heavily unionized IT work environment. People who could barely stand to spend a day with each other in separate cubicles were now stuffed in each other's personal space day in and day out, with threats of impossible deadlines and forced overtime. Perhaps there were a few of employees who were actually IT trained and even fewer of those with a natural ability in the field. Largely the employees were re-trainees over the years who ended up where they were based on seniority alone. And the reward for using this methodology? Cars? Expensive Homes? Vacations in the tropics? No, the reward was more of the same year after year until retirement. Needless to say, the methodology bombed horrifically at this business.

    Never trust anyone quoting best practices...

[reply]
Re: Nobody Expects the Agile Imposition (Part I): Meta Process
by Your Mother (Archbishop) on Feb 01, 2010 at 06:29 UTC
[reply]
Re: Nobody Expects the Agile Imposition (Part I): Meta Process
by bobuillean (Novice) on Feb 02, 2010 at 11:11 UTC

    We have a rigid process forced upon developers. It is so cumbersome that even the most trivial change cannot take less than a man week.

    The justification is "improved quality", the end result is a work environment where innovation is choked off at source.

    I'd like to introduce a process whereby you are tested on your understanding of the work of Robert Glass,DeMarco & Lister, Boehm, Brooks et al before you are allowed your "Process Designer" licence.

[reply]

      I would not be surprised if we were coleagues.

      The resulting codebase is ghastly.

      Jenda
      Enoch was right!
      Enjoy the last years of Rome.

[reply]
Re: Nobody Expects the Agile Imposition (Part I): Meta Process
by Tanktalus (Canon) on Feb 10, 2010 at 17:25 UTC

    Perhaps you're too passionate for your own good ;-)

    Seriously, most of the people I've worked with simply aren't that passionate. Process changes come and process changes go, and very few of us say anything. Agile was introduced by a Big Important Person in my previous position, and, it was basically ignored by management. Oh, not in public, but when us peons asked about adopting some of agile's strengths, management basically said nothing was going to change. Very little complaint.

    My new position is in a team that is allegedly using Agile. Well, iterations anyway. "Short" might be stretching it a bit too far as a descriptor, though - I think my current iteration is somewhere around a month, with future iterations scheduled for 2 weeks each. And their contents have been determined by management, not development. Most developers seem to be just doing what they're told rather than driving for improvement.

    And I can really understand where they come from. Keep your head from poking up, and raising ire (I don't seem to care much about that, which gets me in trouble, both good and bad), get your job done, go home. And pray that they don't change the rules again. Focus on what's important, and for most people, that's not the software. It's the family. Choose your battles wisely, and apparently lots of people have enough battles at home to concern themselves with the battles at work.

    For too many companies, management is about power, not wisdom. And if they allow developers to pick the process methodology, that's like conceding power, and thus inherently counter-productive (to their personal power-base growth). If you find an employer where management is about wisdom, not power, let me know. Especially if they're hiring.

[reply]
Re: Nobody Expects the Agile Imposition (Part I): Meta Process
by kennethk (Abbot) on Feb 01, 2010 at 23:41 UTC
    I'm almost certain I came across this post by way of PM, but for the life of me my search-fu is failing, so maybe it was off Slashdot. Basically, the post says agile works for some folks, and not for others. But it's an enjoyable read.
[reply]
Re: Nobody Expects the Agile Imposition (Part I): Meta Process
by Anonymous Monk on Nov 21, 2014 at 09:23 UTC
[reply]

Back to Meditations