The Science Behind Scrum

Scrum is a flavor of agile software development. In agile the development teams are cross functional and are self-managing. The development cycles, called sprints, are short, 4 weeks or less. By the end of a sprint the code must be functional, tested and working. Scrum functions on something called empirical process control. Traditional software development (command and control) uses defined process control, which is based on the theory of how something should work. This is at the heart of why so many traditional software projects either fail or generate bad code. A defined process control is meant to work on projects that are not very complex, tasks that do not need to be exact, like making hat pins. Empirical process control is used in serious engineering when tolerances need to be exact. 


Instead of planning out your software design months in advance and guessing at what will work, like traditional software development, scrum encourages experimentation via transparency, observation and adaptation. The saying “fail early and often” comes to mind. With scrum one is constantly learning; improving skills, efficiency and quality. Scrum is conducive to creating flow states which leads to mastery. A scrum is a self-organizing team or rather a team without a manager, a team that picks what work it will do and how it will do the work. Whether its creators know it or not scrum has a lot of self-determination theory going on.  Scrum is in many ways intrinsic motivation in sheep’s clothing.

I am floored that I just figured this out. I had heard of agile development and scrum but I had never read anything about it nor have I come across it in any of my research. Scrum is rife with positive psychology, self-determination theory (SDT), behavioral economics and cognitive neuroscience.

None of this is mentioned in any of the literature I have read and by this point I’ve read quite a bit. What is the benefit of using scrums to develop software? Scrum teams develop software up to 15 times faster than traditional project management processes. Holy performance increase batman! Granted a very small number of teams ever get to that level. Gains of 400% are achievable by any team willing to do the work. Not only is the work done quicker but that the end of a development iteration or sprint, the code developed is ready to ship and it generally has less bugs than traditional methods.

I watched a Google Tech Talk: Self-Organization: The Secret Sauce for Improving your Scrum team by one of scrum’s creators: Jeff Sutherland. He talked about how having people co-located creates a social bond. The idea is that if people work closely together they will care about each other and will be a better team and do better work. This comes out of behavioral economics, which tells us that we have two kinds of relationships, social relationships and transactional relationships. Social relationships build trust and result in win-win situations. Transactional relationships build rewards and result in win-lose situations.

30% of all development is being done by agile development teams. Though my research for this post indicates that only 50% of teams that call themselves agile are fully agile and only 10% of scrums are actually scrums. Most teams implement some kind of less successful variant. This kills me! My guess is to truly implement agile or scrum means that management has to change too much, or give up too much power to fully adopt these practices, that or teams are resistant to change. At the end of the day, what matters is speed and quality, not how compliant you are to a framework.

The sacred protector of the team and scrum theory is the scrum master. Part leader, part servant, his job is to ensure the team is fully enabled to succeed and remove any impediment to their success. She does this by being an expert on the framework, genuinely caring about the team and focusing on doing whatever it takes to make the team successful and keep them focused. It certainly would not hurt if the scrum master had a basic understanding of SDT, positive psychology and behavior economics. Though I think all leaders should.

I’m ecstatic to find evidence based practices worming their way into the workplace! As a side benefit it gives me something new to research and write about. Hug your local agile programmer; she is making the world a better place.