I was thinking about some of the challenges with scaling Agile software delivery recently and I took another look at the Spotify Agile engineering model which I hadn’t looked at for some time.
The YouTube video further down this article is a few years old now but it is excellent and still relevant to today’s challenges. I outline some of the key points in this article.
Many of the Agile teams that I have worked with have adopted many of these practices and it is worth taking another look and seeing if your teams can improve their performance by adopting some of them too. It is often the case that what wouldn’t work 6 months ago might work now as people move around between teams, learn more and change their minds.
Before we jump into the scaling Agile part let’s have a quick look at Spotify the company.
Spotify overview
Spotify launched in Sweden in 2008. Today, Spotify is the most popular global audio streaming subscription service with 271m users, including 124m subscribers, across 79 markets. They have 50 million tracks, including 700,000 podcast titles, available to discover, manage and share.
Their development teams are distributed globally so they know a thing or two about managing Agile at scale.
This is what they discovered…
Challenges of scaling Agile delivery
Scaling Agile delivery is challenging. It is quite straight forward to make big strides over the first few years, setup Agile delivery well, and get it functioning effectively. But then what? Where do you go? How far can you go with it? What are the benefits of further improvements and more agility?
Starting with Scrum delivery
Spotify started out using Scrum delivery and this worked very well for a number of years. This gave a nice team-based culture but as the company grew with multiple teams, they found that some of the Scrum practices were getting in the way. So, they decided to make some of them optional.
Agile matters more than Scrum
They discovered that Agile principles mattered more than specific Scrum practices. This was more of a change in mindset, so they renamed some of the elements:
- The “Scrum Master” became an “Agile Coach” to focus on working with teams to improve Agile delivery
- They became “Servant Leaders” rather “Process Masters” there to serve the team rather than stick dogmatically to a process
Squads and autonomy
Spotify replaced the terms “Scrum team” with “Squad”. The key driver became autonomy, and this meant:
- Small cross functional teams of less than 8 people that sit close to each other
- The Squad have end to end responsibility for one or more products
- They do design, build, testing, deployment and operations
- Each Squad has a long-term vision
This means they own the whole thing – so if they build it, they own it and if they break it, they fix it.
Autonomy with boundaries
Autonomy means the Squad decides:
- What to build
- How to build it
- How to work together when building
There are some boundaries that the Squad needs to operate within:
- The Squad’s mission
- The overall product strategy
Agile workspace
You can tell a lot about how Agile teams are by walking around their workspace – this is so important. The workspace setup increases the collaboration for teams:
- Squad members sit close together
- They have easy access to each other
- There is nearby space for retrospectives, planning and problem solving
- All walls are whiteboards
How this works in the changing workplace where more diverse teams are often located in different places, offices and even countries also needs careful consideration (the subject of another video).
The importance of autonomy
Autonomy is important because it motivates people. They are motivated to build better products faster. Autonomy means that teams can make decisions locally and minimise hand-offs between teams so that they don’t get bogged down in dependencies and coordination.
Squad alignment
Although each Squad has a mission they need to be aligned with:
- Product Strategy
- Company priorities
- Other Squads
So, this means:
- They need to be a good citizen in the Spotify ecosystem
- Spotify’s overall mission is more important than one individual Squad
- Be autonomous but don’t over optimise
- Like a jazz band – everyone has their key role
- Loosely coupled but tightly aligned
The Squads experiment a lot with different ways of working now and for the future.
Alignment with other Squads enables autonomy.
Squads, Tribes, Chapters and Guilds
These are some of the other organization and community elements:
- A Squad is a product team owning one or more products
- Squads are grouped into Tribes
- A Tribe is a lightweight matrix
- Chapter is a competency eg. Developer, Agile coaching, DevOps, Testing
- Chapter lead is the line manager of engineers
A Guild is a lightweight community of interest where people share knowledge eg. Leadership, Web Development, Continuous Delivery. Anyone can join or leave a Guild. A Guild has dedicated communication channels and regular meet-ups.
Spotify preference is for community over organisation structure
Role of the company leadership
The company leader’s roles are different too.
Their job is to communicate what problem needs to be solved and why.
The Squad’s job is to collaborate with each other to find the best solution.
Architecture and internal Open Sourcing
Spotify’s aim is to have a systems architecture where components are small and decoupled. They use an internal Open Source model and there is a culture of sharing rather than owning.
Anyone can edit any code even if another Squad technically owns the codebase. So, if a Squad needs some changes to a product or system component and the owning Squad don’t have time to do the change themselves; then the requesting Squad can make the change instead. The owning Squad will simply do a peer review of the code before it can be released.
Small frequent releases
The focus at Spotify is on doing small software releases often. This led, over time, to the reorganization of the architecture so that it is de-coupled to enable independent releases that can be done separately to reduce impact radius of any failures.
Architecture and Squads evolve to enable easier releases
Because the architecture has evolved to enable decoupled releases, then so has the structure of the Squads. So, the Squads evolved to include:
- Client Squads – eg. iOS, Android
- Feature Squads – eg. Search
- Infrastructure Squads – that provide tools to make it easy for other Squads eg. AB testing, CI etc.
Spotify strive for a self-service model and they aim for no handoffs between teams.
Synchronisation of releases
Some coordination of Squad releases is needed. This is achieved with Release trains and Feature toggles.
Release trains are releases that are done at a specific regular frequency or date for example: Every 2 weeks on a Thursday. The release happens every time whether there is much ready for the release or not.
If features or functions are not ready for the release then they miss the release or if they are nearly ready then they can be released as a feature but toggled off. When the code is fully ready for release the release is done and the code is toggled on.
Built on trust
The Spotify model is based on trust so that the development teams have enough autonomy to make good decisions within their boundaries and work across teams to deliver great products.
This is a very brief summary of the Spotify experience of running Agile delivery at scale so have a look at the video too.