In the world of software development, speed is everything. If you don’t deliver fast enough, you can’t compete. The persistent question is, how can my organization truly achieve software delivery at speed and scale? Our answer: with Modern Software Delivery.
Businesses must keep pace with their competitive markets to remain viable. Every organization faces challenges from traditional competitors and new attack vectors to standard innovation and technology disruptors. The ease, availability and ubiquity of technology magnify the frequency and scope of these challenges.
Meanwhile, today’s customers (and employees) are accustomed to highly available applications, constantly functional and consumable anywhere, anytime and on any platform. These forces are causing businesses to constantly re-evaluate their strategies to gain, control and manage their markets. Unfortunately, the way most software and technology is delivered is insufficient to meet these demands.
There is a myriad of internal reasons why IT struggles to keep pace:
- Technical debt is a drag on velocity and increases overall ownership costs
- Monolithic systems, by design, impede agility
- Teams are underdeveloped — they are forced together, resulting in poor productivity
- Long business-case funding and approval cycles bring innovation to a standstill.
The bottom line: Market demand for technology-based solutions creates an ever-increasing need for leveraging technology, which places increasing burdens on IT’s ability to develop and deliver technology.
There is a solution. But first, let’s look briefly at how we got here.
Chapters
The Three Waves of Software Development Modernization
A Holistic, End-to-End View of Software (Product) Development
How You Can Tell if You Suffer From a Traditional IT Value Chain
Better, Faster and More Predictable: A Modern Software Delivery (MSD) Approach
Ensuring Quality Across the Value Delivery Chain
Outcomes, Not Activity – Metrics for Success
Conclusion
The Three Waves of Software Development Modernization
Agile adoption and Development Operations (DevOps) have addressed some of these software delivery challenges, but only in isolation. The massive uptake in the adoption of Agile methodologies — which focus on better integrating the business and IT functions — was the first wave of modernization.
The First Wave of Modernization
In this first wave, organizations were transformed into collaborative and cross-functional teams, helping them improve transparency and provide some adaptability. However, teams did little to accelerate overall delivery velocity. Some organizations experienced longer-term velocity uplift benefits, while others only experience a shorter time for the first feature to market. In fact, according to the 2018 Standish Chaos Report and 13th Annual State of Agile Report, 52 percent of all Agile projects are considered challenged, while another 9 percent outright fail to achieve their cost or overall timeline objectives.
The Second Wave of Modernization
The second wave of modernization began with the advent of DevOps. With DevOps, we saw principles of Design Thinking, Lean and Flow taking hold in the form of automation – specifically, automation pipelines.
Generally, hyper-focus has been on automating specific segments of the overall delivery process, such as CI Builds. But this runs counter to Lean thinking, in which optimizing Flow is the goal. Without addressing the ability to introduce change and size of work, there is a practical limit to how much performance automation can deliver on its own.
The Third Wave of Modernization
In the third wave of modernization, truly effective organizations modernize how they envision, develop, deliver and operate software within the application lifecycle. They transform themselves into collaborative, cross-functional, highly automated, innovative, self-managed, self-serviceable, and productive teams by adopting modern practices such as Value Stream Mapping, Design Thinking, Lean, cross-organizational DevOps, and Lean Agile. The synergy of these multiple modernization approaches represents the third wave of modernization.
Organizations that modeled these proven principles have realized true business value, as seen in improved profitability, productivity and market share. In fact, many organizations that embody these principles see their market share grow by 50 percent more than their peers, per the State of DevOps Report 2017 and DORA State of DevOps Report 2018.
When software development is the center of a business’s strategy, that organization must establish an IT value chain that allows it to stay ahead of the competition by continuously delivering high-quality products at scale and low risk.
This blog will describe how modern software delivery (MSD) organizations structure and further integrate their end-to-end software delivery processes.
For a deeper understanding of MSD and how we can help your company check out our white paper, Rethinking Business Transformation: Thrive Through Disruption with Modern Software Delivery.
A Holistic, End-to-End View of Software (Product) Development
MSD approaches focus on the entire IT value chain, with IT delivery being a system that includes ideation, development, delivery, and operations. What IT professionals do in one area greatly affects what they can do in other areas. Feedback loops are not only necessary for collaboration — they are a natural consequence of any system.
Figure 1: The Modern Software Delivery Pipeline
Figure 1 illustrates the typical components of a modern software delivery organization. All work starts with ideation from the business. Agile management starts by collecting ideas, vetting them, and converting them into a product vision, roadmap, themes and epics.
This work leads to development activities, where Agile teams break epics down into stories and acceptance criteria. Coupled with modular architecture, infrastructure as code, this decomposition of work into the smallest possible units allows an organization to optimize flow and reduce the risk of changes.
MSD’s Holy Grail is the continual flow of “develop a feature, release a feature” in rapid succession. From this point, DevOps automation takes over in the form of continuous integration (CI), continuous delivery (CD), continuous testing (CT) and continuous deployment.
Each of these phases is responsible for building the software, validating it, packaging it for release, and then managing the release of each feature. Automation drives overall velocity, but only when features are small and decoupled. You need sophisticated infrastructure, security and governance to accelerate and achieve optimal velocity at scale and with quality. While this flow looks linear, it is a complex interdependence on multiple organizational capabilities, as outlined in Figure 2 below:
Figure 2: Modern Software Delivery Capability Interlocks
The relationship between several factors regulates the ability to increase speed and optimize flow:
- The size of each work unit
- Transparency in a work unit’s current state
- Automated handoffs and flows
- Automated provisioning
- Movement of work through the organization.
While automation is important, the real work is synchronizing multiple types of tasks in that automated flow while also reducing risk when changing an existing system. When a change breaks the software, you need to remove the offending change to maintain a clean work product. Sophisticated organizations can “unwind” work all the way to the point of origin – including the associated code commit if needed.
For example, if a developer checks in a piece of code that fails after being deployed to production, a sophisticated system will roll back to the last known good version. The system will completely mark every work stage that piece of code previously passed as a “failure” – going as far as removing the bad code from the codebase. Presumably, the right quality checks exist at each step, so such a rollback should be an infrequent occurrence.
Closing this feedback loop requires actively monitoring end-user usage and gathering feedback to inform prioritization and product exploration. Focus on the end-to-end development process, from product ideation to customer value, is critical to establishing a vision for a modern software delivery organization.
Depending on the organization’s ambition, this flow of activities can take weeks or mere hours. This is modern software delivery at speed and scale. The key question is, how fast does your organization need to go?
As I like to say, “What works for Twitter doesn’t work for NASA, and vice versa.” The level of investment in a sophisticated system needed to match the speed of your business needs and industry will affect where and what you should invest in. This is a great opportunity to reach out to an expert to help determine what you need and what it’ll take to get there.
How You Can Tell if You Suffer From a Traditional IT Value Chain
The key symptom that an organization has a legacy approach to development and delivery is the business department’s relationship with IT.
Often, strained relationships between IT and the business result directly from the business’s need for ever-increasing demands for agility and speed and IT’s inability to match the pace of demand. This leads to mismatched expectations and frustration.
When faced with chronic dissatisfaction, businesses will reach outside their organization for a solution. The ubiquity and ease of access to ready-on-demand, targeted solutions — especially with many cloud or software as a solution (SaaS) approaches being a credit card sign up away — means that businesses will happily and readily bypass IT departments. This creates unmanaged and decentralized IT solutions – it creates shadow IT.
Failure to achieve predictable delivery within budget is another hallmark of a traditional IT value chain. Even organizations that have transformed to Agile practices can struggle to achieve predictable delivery. Often, you will see these organizations have a hyper-focus on velocity and epic burndowns rather than outcome or value. When you see this, you must step back to look at the IT value chain to see how you are delivering value (and not just features) to the business.
When delivery of themes and epics are the focus of an organization, you typically see them revert to a linear process where they are waiting for value at the end of the process after an accumulation of work has transpired.
With Scrum, this means waiting for value for up to 2 weeks – due to the nature of accumulating stories in an artificial 2-week time box and then releasing them as a batch. Two-week sprints innately shortened the time to achieve some value, compared to traditional waterfall approaches, but many organizations cannot, or will not, release software every two weeks.
The inability or unwillingness to release as soon as they complete their work means the organization cannot derive value from what they created. This actually is a form of technical debt – investment in work that cannot create value. While better than the old waterfall environment, it still directly impacts organizations and impedes achieving the full benefits of Agile and DevOps.
Better, Faster and More Predictable: A Modern Software Delivery Approach
Our proven MSD methodology enables software delivery that provides reliable, scalable solutions for our clients. But we also teach your teams to adopt our MSD approach. That empowers them to build and deliver modern software themselves, synergizing Agile management, effective test automation, DevOps practices, modular architecture and cloud platforms. By looking at IT as a system and living organism, we can address bottlenecks as they arise and ensure we target investment at current challenges while building toward a future vision.
Figure 3: Application Development
Unfortunately, organizations often target one area at a time without regard to its impact on other areas.
For example, many organizations have undertaken an Agile transformation, and some have even implemented SAFe. As a result, most teams are operating in Scrum fashion. However, as those same organizations look to implement DevOps, the size of the story directly affects the speed at which they can perform things like CI. After all, you can’t integrate a story faster than you can develop it, and you can’t release faster than you can develop the story.
Even still, as most organizations mature with Agile and transition to implementing DevOps, they often encounter their next bottleneck: testing. While the Cloud is DevOps ready, your operations may not be DevOps capable. Only by focusing on MSD as a system can your organization truly achieve the promise of many of these practices and methodologies.
Ensuring Quality Across the Value Delivery Chain
Regardless of where you are in the software development life cycle — from defining your product through development, delivery and operations — you must create an environment for quality across the value delivery chain. After all, quality is an attribute of what you build. This simple concept has far-reaching consequences for anybody involved in delivering IT value.
To achieve quality at speed and scale, it’s important to:
- Define quality objectives
- Drive quality thinking across the value chain
- Measure quality.
Because quality is an outcome and not a phase in the process, you must define what you expect, what is acceptable, and what is not at each step in the IT value chain.
For instance, is it acceptable to build a product without a defined product vision or roadmap? What about without a defined or expected business outcome? What about at the story level – should you develop stories that don’t have acceptance criteria?
As you drive through the value chain, the specific questions may change, but not their nature. For example, at the developer/tester level with CI/CD/CT practices, quality is more often an inspection idea, such as “all stories must be tested with unit, system and integration tests.”
However, what engineering practices are you following to minimize the introduction of coding issues? Are you using linting tools? Code-scanning tools? You must drive quality throughout the entire chain, from ideation to development to delivery to operations. Failure to drive preventative and inspection activities will always result in churn in your process, dragging on overall velocity as you react to issues instead of actively preventing issues.
Outcomes, Not Activity – Metrics for Success
Agile Development Teams using DevOps automation pipelines must measure outcomes, not activities. If you don’t have a clear picture of the results that matter, you can’t know if you were successful. Agile Development Teams need to define what metrics will drive project success and create development processes to support them. You measure success by outcomes, not activities.
To this end, each step of the IT value chain has an associated set of outcome metrics that are valuable to your organization. These include metrics applicable to build automation, such as build success percentages and build-time SLA percentages. For QA and their test suite, this might be test automation percentage coverage. Deployment pipelines would include deployment downtime SLA percentages or successful automated release percentages. For product management focusing on predictable delivery, this might be lead time or cycle time.
Successful metrics for your delivery process will combine leading and lagging measures that focus on what you achieved, not what you did. No one cares how many nails you use to build a house, but they care that it is well built. Similarly, you must focus on value delivered to customers, flow and early identification of bottlenecks and constraints.
Conclusion
Delivering software at speed and scale is essential in today’s business environment. By adopting an MSD approach, you can combine Lean-Agile thinking, DevOps automation pipelines, Cloud-ready platforms with modern software architecture practices. The result is a holistic, end-to-end IT value chain that delivers software better, faster and more predictably. Armed with a modern delivery methodology, you will be better able to adapt and deliver software ready to meet your customers’ and employees’ needs.
