http://qs321.pair.com?node_id=820672

Chapman: One of the cross beams has gone out askew on the treadle.
Cleveland: Well what on earth does that mean?
Chapman: I don't know -- Mr Wentworth just told me to come in here and say that there was trouble at the mill, that's all -- I didn't expect a kind of Spanish Inquisition.
Ximinez: NOBODY expects the Spanish Inquisition!

-- The Spanish Inquisition by Monty Python

As the Monty Python boys might say: Nobody expects the Agile Imposition! At least, I didn't. Though I noticed some of the cross beams of our development process had gone out askew on the treadle, I only expected some minor adjustments to our existing process -- not an agile imposition.

I was also taken aback at the way it was introduced. With little prior consultation or discussion, all development teams were instructed by management to follow the Scrum development process, effective immediately.

Imposing a process on a team is completely opposed to the principles of agile software, and has been since its inception... An important consequence of these values and principles is that a team should choose its own process -- one that suits the people and context in which they work.

-- Martin Fowler

As for the rationale for this Scrum imposition, development managers are typically under intense pressure to improve productivity. Accordingly, our management team is always on the lookout for process improvements. Scrum was successfully tried on a pilot project, so there was a natural urge to apply it more broadly.

Please remember that just because a technique worked for you last year and for one project, it does not follow that it will work unmodified for someone else or for a different project.

-- Bjarne Stroustrup, The C++ Programming Language, Chapter 23, Development and Design

Flushed with the initial success of the pilot, there may have been some unrealistic expectations that this new process might finally deliver the long sought after order-of-magnitude productivity boost. Yet software development remains hard. There is no silver bullet.

What disturbed me most about this little episode was there seemed to be no place for common sense in this well-intentioned attempt to improve the way things are done. When I suggested team-specific adjustments to their process, it was implied that I "didn't get it" and that I'd benefit from attending another Scrum training course, maybe one day even attaining the coveted "Certified Scrum Grand Panjandrum" title. I felt sad and alone, a lowly novice in this new religious order. Most developers, and all managers, seemed to disagree with my points of view.

For therapy, and as a reality check, I'm writing this new series of articles. I encourage you to tell me where my cross beams have gone askew on the treadle. Your feedback, anecdotes, opinions, advocism, and any citations you may know of, are welcome.

Software Development Methodology "Science"

This type of experimental design seems to be missing from current empirical software development research. Many will argue that such experiments are impossible, or unreasonably expensive, when studying how highly paid people perform lengthy complex tasks. That may be true, but it doesn't justify misrepresenting anecdotes as scientific results.

-- David Parnas in IEEE Software Magazine, Nov/Dec 2009 (page 58)

I'm generally skeptical of software development methodology "science". Stronger, I'm profoundly skeptical of claims like "we measured a 400% productivity improvement by switching to the amazing new XYZ development methodology!!". Apart from the obvious conflict of interest in measuring the productivity of a methodology you're making money from, performing anything resembling a valid scientific experiment comparing development methodologies is dauntingly difficult and often prohibitively expensive. After all, how on earth do you "prove" that a methodology that "worked" for one team on one project will similarly "work" for a different team on a different project? And how can you "prove" that the original team would have performed the said project better (or worse) had they adopted a different methodology? Lacking a time machine, you can't just re-run the experiment. The best you can hope for are fuzzy averages and statistics for your organisation. Which may or may not apply to different organisations. It's a very difficult problem.

Should Teams be Allowed to Choose Their Own Process?

It turns out that there is no one way to motivate people, no best way to organize teams, and no universal rule governing interactions. In addition to cultural differences, each individual has unique strengths, styles of learning, and values.

-- Leading Lean Software Development by Mary and Tom Poppendieck (p.187)

Each project's ecosystem is unique. In principle, it should be impossible to say anything concrete and substantive about all teams' ecosystems. It is. Only the people on the team can deduce and decide what will work in that particular environment and tune the environment to support them.

-- Communicating, cooperating teams by Alistair Cockburn

It's never a good idea to force someone to practice XP against his will.

-- The Art of Agile Development by James Shore & Shane Warden (page 47)

I'm interested to learn:

Notice that competent individuals, and teams, are refining and improving their processes all the time. I contend that it's better for process improvements to be driven by individuals and teams -- rather than being mandated from above -- because this improves morale and motivation (via empowerment) and makes it clear that improving process is everybody's job and that it needs to be performed constantly and forever. For what it's worth, I understand that both Google and Microsoft allow each development team to choose their own process.

People versus Process

Generally, I feel it's better to focus more on people than process. What do you think?

The closest I could find to "scientific" support for my point of view is:

No shortage of anecdotes though:

Given the importance of motivation, I'm interested to learn what motivates you, what are good ways to motivate and reward programmers.

Individuality

It's the team that matters. Where would The Beatles be without Ringo? If John got Yoko to play drums the history of music would be completely different.

-- David Brent

Another point I'm interested to hear your opinion on is the popular management notion that programmers are "equal and interchangeable". While making projects easier to "manage", I feel this fallacy harms the motivation of programmers who take great pride in their work. At least, I gain much more joy from becoming an expert than from performing an "average interchangeable" job with little regard for quality or craft.

Managing Change

And it should be considered that nothing is more difficult to handle, more doubtful of success, nor more dangerous to manage, than to put oneself at the head of introducing new orders.

-- Niccolo Machiavelli, The Prince (1513)

The fundamental response to change is not logical, but emotional.

-- from Peopleware, Chapter 30, Making Change Possible

This is much harder than it looks. Many people, especially older folks like me, naturally resist being changed. Transforming a productive and proficient oldbie into an ineffective newbie doesn't seem like a good idea to me. It's vital to explain to people why they should want to change, and so cultivate readiness, not resistance.

Process Religion

Scrum as an institutional change agent is invaluable to a church.

The law in Scrum that nothing new can interrupt a sprint plays havoc with the life of a minister ... unanticipated short-term emergencies happen every day! People die. Pastoral emergencies emerge. These cannot be scheduled.

Vocabulary can be a stumbling block. Many church people associate words such as "product owner" and "value added" as tools of a business or corporate environment that are at best suspect. "We are a church," they protest, "not Microsoft or a bank!" I have experimented with "vision holder" to describe the product owner.

-- from Scrum in Church co-authored by Scrum co-founder Jeff Sutherland

The final section of this first episode notes a lamentable aspect of human nature: well-intentioned process zealots who get way too excited about their methodology. Here's an example taken from the book Agile Project Management with Scrum by Scrum co-founder Ken Schwaber.

The Sprint felt dead. As ScrumMaster, I could have called for an abnormal termination of Sprint. Circumstances had changed so that the Sprint goal appeared to be unobtainable. I just couldn't do this, though. WebNewSite had just started with Scrum; I had told them that the ScrumMaster removed impediments. This was certainly an impediment -- Thomas couldn't be reached. The more I thought about it, the more I felt that this was unacceptable. Even though Thomas hadn't left a phone number, I was sure that he wouldn't want to leave the team in the lurch and that he would want to immediately come to the team's aid if he only knew of its predicament. But how could I contact him?

Unfortunately for Thomas and his vacation plans, I'm a fan of mystery novels. "Of course!" I thought. I'll hire a private detective to find Thomas. After some searching, I found an ex-FBI agent who had an office in Billings, Montana. He was excited about working for an Internet startup. He found Thomas within two days. Thomas was able to assist the team, and the Sprint goal was met. I figured that I had been pretty inventive, spending only several hundred dollars to get around a major impediment in an unconventional manner.

-- from chapter eight of Agile Project Management with Scrum by Scrum co-founder Ken Schwaber

Though Ken seemed proud of his unorthodox antics, divulging an employee's social security number to a private detective so that he can locate him and drag him back from vacation to work on a Scrum sprint seems like a violation of common sense to me (and may even be a violation of common law). Certainly, I would feel violated were this done to me. What do you think?

It is easy for an individual or an organization to get excited about "doing things right"... common sense can be the first victim of a genuine and often ardent desire to improve the way things are done. Unfortunately, once common sense is missing there is no limit to the damage that can unwittingly be done.

-- Bjarne Stroustrup

Other Articles in This Series

References

References Added Later

Updated 3-feb: Minor improvements to wording. Added finishing Stroustrup quote.

Back to Meditations