Tips for Avoiding Local Optimization in Your Agile Process

HOSTING recently sponsored a special session of  the Agile Denver Scrum Master Guild for a presentation by renowned author Craig Larman. The Scrum Master Guild is run by volunteer members, and provides coaching and education for Scrum Masters in the Denver metro area under the umbrella of Agile Denver.

We were lucky to get a special session with Mr. Larman while he was in Denver providing training on LeSS – the Large Scale Scrum framework. Craig took us on an interesting journey, discussing the history of the agile movement, best practices for implementing an Agile process, and some of the driving forces behind the development of LeSS.

The best laid plans of mice and men

A large portion of Craig’s presentation was focused on local optimization. Local optimization frequently occurs in larger companies when areas of specialization are identified, and groups of specialists are developed to address them. This frequently manifests in the form of component teams.

Optimizing the development teams to work on specific components certainly looks and feels like an excellent idea at first glance – however, it can come with challenges. Craig described some of the negative and unintended consequences of local optimization including:

  • Private code – Functional areas become so tied to the component team that other teams are unable to share code. Component teams often end up creating complex ‘feature branching’ schemes that further complicate the private code.
  • Resource competition – Inevitably, projects require coordinated efforts in order to achieve success. Forcing development teams to switch context frequently leads to wasted time and lost efficiency. It can result in long-lived backlogs and differing priorities within each team, which lead to longer cycle times.
  • Layered requirements analysis – Siloed teams make it easy to lose track of the “big picture” (think 2nd grade telephone game!)

Craig described his involvement with a project that had a large number of component (locally optimized) teams, all feeding an “integration team.” The integration team wasn’t able to pull together the inputs from many locally optimized teams, and an equally large number of branches, to create a functional system. As a result, the project failed, losing 50 million dollars in the process. Scary!

Craig’s story highlighted that the best intentions of local optimization can hide detrimental systemic issues at play. In the eyes of those responsible, a locally optimized system can be considered “most efficient” or “most productive,” even though it is introducing systemic communication, prioritization, context switching and integration problems.

Craig provided examples of how various Agile practices address the issues of local optimization including:

  • Feature teams:
    • Avoid specialization – the team is responsible for the entire product, and its implementation
    • Provide intense product focus with customer feedback
    • Reduce context switching, as the team retains focus on the current feature under development
  • Continuous integration – no ‘Big Bang’ integrations ($50M failures!)
  • Iterations (sprints) provide frequent opportunities for feedback, learning and adaptation.
  • Visibility to customers and management – no surprises and awareness of evolving product shape
  • Lean principles strive to reduce the cycle time of product delivery  and incremental changes to it
  • LeSS – Large Scale Scrum – provides a mechanism for coordination of large products consisting of many teams

There IS a light at the end of the tunnel

It is easy to draw parallels between intense presentations such as Craig’s, and product development.

A common pattern of local optimization is to separate teams for UI (user interface) and backend work of a product. This pattern seems pretty safe and often works quite well. However, it is very fragile. Introducing differing priorities across the teams may result in the backend work being completed significantly ahead of the UI activities. When the UI team starts work they discover that the backend API does not provide the required data. By this time, the backend team has moved on to different work and their response will be delayed, as they have to switch context away from their new priority. Ultimately, this project ends up being deferred, as higher priority work has been identified and pulling the backend team away from it is not considered worth the additional investment. UI and backend “component teams” are a form of local optimization.

By contrast, the HOSTING development team is currently working on integrating management of SRX firewalls into our customer portal. The scope for this project was broken down into multiple features, with our minimum viable product consisting of visibility into the current firewall rules within the portal. The team working on this effort includes both frontend and backend developers, along with a dedicated product owner and designer, and heavy involvement from our internal network operations team. With ongoing collaboration, the team has been able to gather extensive feedback from both customers and internal stakeholders, turning out a working product within just a few weeks.

Development teams commonly struggle with aligning schedules and priorities, sharing code and knowledge between components, and delivering products that truly meet customers’ needs. Following are examples of the Agile practices in place at HOSTING that align with Craig’s recommendations to counter local optimization.

  • Periodic sprint reviews provide constant visibility into our work in progress – not just within each team, but across the organization. We clearly see the impacts of introducing an additional feature into a team’s ready queue. Our leadership is developing strong cognizance of these impacts, and as a result, approaches the introduction of new features very carefully.
  • We have introduced dedicated product owners into the teams, to ensure effective communication between the developers and our customers and end users. The close feedback gained through this approach has allowed our beta customers to shape our products more closely to their needs, ensuring that we’re solving real customer problems, and providing the most value from the customer’s perspective.

The applications and teams that HOSTING builds are nowhere near the complexity of the 50 million dollar failure that Craig described, but even at much smaller scale, we have anticipated and addressed potentially negative consequences of local optimization. Product development is an ongoing process, and through agile practice adoption, especially continuous improvement, has reinforced HOSTING’s position as a lean, agile, globally-aware organization.


About the Author

Wade Scherer

Wade Scherer

Wade is Agile Coach and Scrum Master for HOSTING. He has worked in the software development arena for more than three decades, continually applying and advocating for best agile practices. Wade is passionate about leveraging new development and management techniques to help improve team performance and product quality.

Leave a Reply

Your email address will not be published. Required fields are marked *