Scrum vs. XP: what if Scrum was right? (sort of)

So here's a question. As an XP person, my consulting leaned into saying people should for real try all of XP for a while before passing judgment. I think that was fairly common, back in the day.

The Scrum approach was different in two ways. First, they were clever enough to realize they could get rich through the "certified Scrum Master" scam. I'm going to leave that aside. The other way is they didn't make teams do the hard technical stuff.

I think the Scrummification of Agile has been a disaster, turning it into an empty management fad. However, what if Scrum was right in its approach? Not the scam stuff but the approach to technical change.

Here's the thing. I'm preparing a third podcast episode on Scott's Seeing Like a State. In it, I'm pointing out that the common notion that people are change-resistant is wrong, or at least too crude.

Scott says, “traditional peoples will embrace techniques that solve vital problems. Sewing machines, matches, flashlights, kerosene, plastic bowls, and antibiotics are only a tiny sample of the products that solved vital problems or eliminated great drudgery and were thus readily accepted.”

Moreover, traditional people were very much into process improvement, but of a special sort. Speaking of Andean potato farmers, Scott says, “The variety of cultivars makes for local experimentation with new crosses and hybrids, each of which is tested and exchanged among farmers, and the many landraces of potatoes thus developed have unique characteristics that become well known. From the appearance of a new variety to its substantial use in the fields takes at least five or six years. Each season is the occasion for a new round of prudent bets, with last season’s results in terms of yield, disease, prices, and response to changed plot conditions having been carefully weighed. These farms are market-oriented experiment stations with good yields, great adaptability and reliability.”

This process improvement is slower than we might like, but what I think is more interesting is that it's deliberately gradual and continuous. It's a hill-climbing algorithm, inherently conservative.

That suggests to me that we – I – blundered in introducing XP. I think it's clearly true that the most successful part of XP has been continuous integration. It seems like everybody these days hooks their version control system to a server that builds it and runs all the tests. That works well with many styles.

Probably, when I was introducing XP, I should have started with that.

One of the thing that most appealed to me about XP was the connections between the practices. I often told people that pair programming supported both TDD and refactoring. My pitch was like this. If you're working by yourself and you know you ought to write some tests or refactor the code you've found yourself working with, it's easy to not do that. But if you're pairing, "not doing that" means that I look at you, you look at me, and we both agree to cheat. Turning an individual decision into a social decision makes it harder to decide to do wrong.

But that suggests it's mistaken to introduce TDD, refactoring, and pairing all at the same time. Imagine we were like Andean potato farmers, taking our time to decide how to change. That might suggest this sequence of introducing XP:

1. We won't do pair programming, but let's do pair reviews of pull requests. Before making a pull request that everyone might review, find someone to give a personal review (in person, or through something like Zoom).
2. I assume the review will lead to revision. (Frankly, more likely to lead to substantive revision than an asynchronous review. Asynchonous reviews, in my experience, focus on trivial things.) Gosh, wouldn't it have been better to catch that earlier?
3. That suggests an experiment: try having you and the reviewer write the original code together. How does that feel? Did you avoid the need to rewrite because problems were detected faster? Was the code better? Did you learn anything?
4. If pairing becomes at least intermittently popular, now introduce refactoring as something pairs might do.
5. If that works, introduce TDD as something pairs might do.
6. Etc.

It used to be a commonplace among XP and Scrum people that Scrum's self-organizing teams would naturally come to adopt XP practices. That didn't work. I suspect that's because it assumed that people would rationally choose to experiment with XP practices. But I think people don't work like that, at least not often enough. A gradual introduction of XP practices might work better.