The failure of the Spotify model

There are articles like this that upset received ideas a little and allow us to take a step back.

This is the case of " Spotify's Failed #SquadGoals » published on 04/19/2020 by Jeremiah Lee which describes from the inside the difficulties encountered by Spotify in the implementation of its own organization. He invites us (like many others before him) not to imitate them but above all to learn from them, which he generously shares with us, as well as many additional resources.

Here is a translation to allow a slightly wider distribution in the French community, while keeping in mind that there is no miracle solution but that its research can lead us to beautiful discoveries and beautiful stories. Good reading !

Note: original post is licensed Creative Commons Attribution-ShareAlike 4.0 International license. So full credit goes to Jeremiah for his work.

Beginning of translation

Spotify doesn't use the "Spotify model" and neither should you.

Of all the attractions of startup culture, few are more desirable than the speed and agility of a small team. Maintaining this feeling in a growing society is a challenge. In 2012, Spotify shared how they worked and suggested they had succeeded.¹

I was excited to see the spotify template in action when I interviewed for a product management role at their Stockholm headquarters in 2017. However, the recruiter surprised me before the first interview. She warned me not to expect Spotify to be a utopia Agile.

I joined the company after it had tripled in size to 3000 people in 18 months. I learned that the famous squad model was only an ambition and was never fully implemented. I witnessed organizational chaos as company leaders transitioned incrementally to more traditional management structures.

When I asked my colleagues why the content hadn't been removed or updated to reflect reality, I never received a good answer. Many people ironically thought articles were great for recruiting. I no longer work at Spotify, so I'm sharing my experience to set the record straight. Spotify's squad model failed Spotify and it will fail your company too.

But you don't have to take my word for it

The co-author of the Spotify model² and multiple agile coaches who worked at Spotify have been telling people not to copy it for years. Unfortunately, the truth does not spread as quickly or as widely as an idea people want to believe.

Even when we wrote it, we weren't doing it. It was half ambition, half approximation. People really struggled to copy something that never really existed. 

—Joakim Sundén, agile coach at Spotify (2011–2017)

It worries me when people look at what we do and think it's a framework they can just copy and implement. … We really try to emphasize the fact that we have problems as well. It's not “glittering like everything is working fine and all of our squads are super fantastic.

—Anders Ivarsson, Spotify whitepaper co-author³

Summary

You can read and watch the original content in less than 30 minutes or skip to the next section if you are already familiar with it. Here is a brief summary.

Spotify had teams called squads because it sounded better (no kidding). A group of teams were organized in a department called tribe (tribe). Each team was planned to be a self-contained mini-startup, with a product manager acting as a mini-CEO for their area of functionality. The teams had designers and software engineers with a range of specializations. The intent was that a team should have all the necessary skills without needing to rely on another team to succeed.

Product managers had a traditional management structure. A product manager for a team reported to the product director of their department (“tribe lead”). Same for designers. Software engineers, however, were managed outside of the team structure.

THE “Chapter leads” managed software engineers specializing in a specific type of software development within the department. For example, all software engineers working on backend APIs in all teams within the department would have one manager and all mobile Android engineers in the department would have a different manager. The intention was to allow engineers to be moved between teams within the department to best meet business challenges without having to change managers.

Spotify tried a long-lived team matrix structure with some unusual word choices.¹ It didn't work well.

Why it didn't work

1. Matrix management solved the wrong problem
2. We focused on the autonomy of the team
3. Collaboration was a supposed skill
4. The mythology became difficult to change

1. Matrix management solved the wrong problem

The full-stack agile team worked well, but matrix management of software engineers introduced more problems than it solved.

• The teams at Spotify were pretty long-lived. The benefit of not having to change managers when moving to another team was limited.

• Engineering managers in this model had little responsibility beyond developing the careers of the people they managed. Even then, their ability to assist in the development of interpersonal skills was limited. An engineer's manager would not manage the other people on the team enough or be involved enough in the day-to-day context to assess conflicts within the team independently.

Without a single engineering manager responsible for the engineers on the team, the product manager lacked an equivalent peer – a mini-CTO for their mini-CEO role. There was no one person responsible (accountable) for the production of the engineering team or who could negotiate the prioritization of work at an equivalent level of responsibility.

When a disagreement within the engineering team arose, the product manager needed to negotiate with all the engineers on the team. If the engineers couldn't reach consensus, the product manager needed to escalate to as many engineering managers as there were engineering specializations on the team. A team with backend, web application and mobile application engineers would have at least 3 engineering managers to involve. If these engineering managers could not reach consensus, a single team issue would have to escalate to the director of the engineering department.

Chapter leads are servant leaders who help you grow as an individual. They don't really work with teams. They have subordinates in all the teams. They don't really have any accountability for production. They don't take that responsibility. It's easy to see the product owner as the manager of the team.

—Joakim Sundén, agile coach at Spotify⁴

Learn from Spotify Mistakes: A product team (designers/engineers) typically contains more engineers than designers or product managers. Having a single engineering manager for the engineers on the team creates an escalation path of responsibility for conflict within the team. Product managers should have an equivalent peer for engineering. Product managers should be responsible (accountable) for the prioritization of work. Engineering managers should be accountable for engineering achievement, which includes being able to negotiate speed and quality trade-offs with the product manager.

2. Spotify focused on team autonomy

When a company is small, teams must process a wide range of work to produce and must shift initiatives frequently. As it grows from a startup to a scale-up, duplicate functions in teams move into new dedicated teams to increase organizational efficiency by reducing duplication. With more teams, the need for an initiative shifter team decreases in frequency. These 2 changes allow teams to think deeper and longer term about the problems they are prone to solve. Faster iteration, however, is not guaranteed. Every responsibility a team gives up to improve focus becomes a new cross-team dependency.

Spotify has not defined a common process for cross-team collaboration. Allowing each team to have their own mode of operation meant that each team needed a unique mode of engagement to collaborate. The overall productivity of the organization suffered.

The Spotify model was documented when Spotify was a much smaller company. It was planned to be a multi-part series, according to Anders Ivarsson. Autonomy had made the first cut, but the other parts on alignment and accountability were never finished.

Learn from Spotify Mistakes: Autonomy requires alignment. Business priorities must be set by leadership. Autonomy does not mean that teams can do whatever they want. Processes for cross-team collaboration should be defined. Autonomy does not mean letting the teams organize themselves for each problem. How success is measured needs to be defined by leadership so people can effectively negotiate inter-team dependency prioritization Autonomy requires accountability. Product management is responsible (accountable) for the value. The team is responsible (accountable) for the production of "completed" increments. Mature teams can justify their interdependence with their ability to articulate business value, risk, learning, and their optimal next move.6

If I had to do one thing differently, I would say that we shouldn't focus so much on autonomy.

Every time you have a new team, they have to reinvent the wheel on how they should work. Maybe, just maybe, we should have a “minimum viable agility”. You start with that. You're free to get out, but people shouldn't have to get out all the time.

When do you start inserting this process? Probably when it's too late.

—Joakim Sundén, agile coach at Spotify⁴

Henrik Kniberg was talking about how we weren't good at big initiatives and we're still not very good at big initiatives.

If you have inconsistent ways of working, it's harder for people to move. If it's harder for people to move, you're more likely to have inconsistent ways of working. It's just going to get stronger until all of a sudden you're not working for the same company anymore. You work for these kinds of weird subcultures.

—Jason Yip, agile coach at Spotify (2015–date of writing)⁵

3. Collaboration was a supposed skill

While Spotify gave teams control over how they worked, many people lacked the basic knowledge of Agile practices. This resulted in teams iterating on process tweaks in a blind hope of finding the combination that would help them improve their production. People lacked a common language to effectively discuss process issues, instructions for solving them, and the experience to evaluate performance. It wasn't very agile. It wasn't fair steps from the waterfall (not-waterfall).

The “coaches Agile" were internal consultants that Spotify provided to teams to teach and suggest process improvements. While the intentions were good, there were not enough coaches to help each team. A coaching engagement with a team was rarely long enough to last until the end of the project to help the team assess its performance. Moreover, they were not responsible (accountable) for anything.

Control without skill is chaos.

-I. David Marquet, Turn the Ship Around!

Learn from Spotify Mistakes: Collaboration is a skill that requires knowledge and practice. Managers should not assume that people have an existing understanding of Agile practices. When a company becomes large enough, teams are going to need dedicated support to guide planning within the team and structure collaboration across teams. Program management can be accountable for the planning process. Dedicated program managers catalyze teams the same way product managers and engineering managers do with their respective skills.

4. Mythology is hard to change

When Scrum introduced new meanings to a bunch of words like burn down and sprint, he did this because it introduced new concepts that needed names. Spotify introduced the vocabulary of missions, tribes, squads, guilds, And chapter leads to describe his way of working. It gave the illusion that they had created something worth learning unusual word choices.

However, if we remove the unnecessary synonyms from the ideas, the Spotify model turns out to be a set of teams cross-functional with too much autonomy and a poor management structure. Do not get fooled. If Spotify had referred to these ideas by their original names, perhaps they could have assessed them more fairly when they failed rather than having to grapple with changing their cultural identity just to find internal processes that worked. GOOD.

Learn from Spotify Mistakes: Most companies can only maintain a certain number of areas of innovation. An internal process is rarely the first area of innovation that differentiates a company in the marketplace. The study of the past allows companies to choose better areas of innovation. Optimize for knowledge. Every new thing a person needs to learn to be productive in your organization should be assessed on its value. Terms Business units, departments, teams, And managers communicate the roles and responsibilities of the organizational structure more effectively than Spotify's synonyms and are not attached to a way of working that failed for its creator.

Do this instead

(Just kidding. There is no magic bullet.)

You may have discovered the Spotify model because you were looking for how to structure your teams. Don't stop there. Keep looking. Company leaders who have withstood longer test times have written far more than Spotify has blogged. Humans have been trying to figure out how to work together for as long as there have been humans. The Industrial Age and the Information Age have changed some constraints, but researchers studying organizational theories have found timeless truths about what humans need to succeed collectively.

It turns out that Spotify hadn't figured out in 2012 how to maintain the speed and agility of a small team in a large organization. The company evolved beyond its eponymous model and looked outside itself for better answers. You should too.

Some of my recommendations related to the topics covered by the Spotify way of working:

• More than 200 people in your product organization? THE Scaled Agile Framework worked well for Fitbit when I worked there.
• Less than 200 people? shape-up by Basecamp is how I intend to structure my next startup.
• Essential Scrum by Kenneth S. Rubin
• Team Topologies by Matthew Skelton and Manuel Pais

Notes and quotes

¹: Scaling Agile @ Spotify white paper, Spotify Engineering Culture video

²: Anders Ivarsson and Henrik Kniberg, authors of the white paper “ Scaling Agile @Spotify ». Henrik clarified its creator status in 2015: “People sometimes seem to make the assumption that I invented the Spotify model. Well, that's definitely not the case! I am only the messenger. … The Spotify model is the result of many people's collaboration and experimentation over time, and many aspects of the model were invented without my involvement. I certainly wouldn't want to take credit for those involved."

³: Episode 112: Inside Spotify with Anders Ivarsson, The Agile Revolution, 2016

⁴: You can do better than the Spotify model by Joakim Sundén, 2017 video, slides

⁵: How things still don't quite work at Spotify and how we're trying to solve it by Jason Yip, 2017 video, slides

⁶: Balancing Autonomy with Accountability by Edwin Dando

Additional Resources

If 2,200+ words of first-hand experiences from 4 Spotify employees weren't enough, read how the Spotify model failed for those people outside of Spotify.

• How to Structure an Engineering Team for Scale by Yotam Hadass
• You want to adopt the “Spotify Model”? I don't think it means what you think it means! by Willem-Jan Ageling
• Spotify sucks! by Erwin Verweij
• Don't do the Spotify Model! by Udhesh Vivekanandan
• There Is No Spotify Model for Scaling Agile by Anthony Mersino