Over the past 25 to 30 years, agile innovation methods have greatly increased success rates in software development, improved quality and speed to market, and boosted the motivation and productivity of IT teams. Now those methods are spreading across a broad range of industries and functions and even reaching into the C-suite. But many executives don’t understand how to promote and benefit from agile; often they manage in ways that run counter to its principles and practices, undermining the effectiveness of agile teams in their organizations.
From their work studying and advising companies that have successfully employed agile methods, the authors have discerned six crucial practices for capitalizing on agile’s potential: (1) Learn how agile really works; (2) understand when it is appropriate; (3) start small and let passionate evangelists spread the word; (4) allow teams that have mastered the process to customize their practices; (5) practice agile at the top; and (6) destroy corporate barriers to agile behaviors. They expand on each, providing executives with a practical guide for accelerating innovation and profitable growth.
Agile innovation methods have revolutionized information technology. Over the past 25 to 30 years they have greatly increased success rates in software development, improved quality and speed to market, and boosted the motivation and productivity of IT teams.
Now agile methodologies—which involve new values, principles, practices, and benefits and are a radical alternative to command-and-control-style management—are spreading across a broad range of industries and functions and even into the C-suite. National Public Radio employs agile methods to create new programming. John Deere uses them to develop new machines, and Saab to produce new fighter jets. Intronis, a leader in cloud backup services, uses them in marketing. C.H. Robinson, a global third-party logistics provider, applies them in human resources. Mission Bell Winery uses them for everything from wine production to warehousing to running its senior leadership group. And GE relies on them to speed a much-publicized transition from 20th-century conglomerate to 21st-century “digital industrial company.” By taking people out of their functional silos and putting them in self-managed and customer-focused multidisciplinary teams, the agile approach is not only accelerating profitable growth but also helping to create a new generation of skilled general managers.
The spread of agile raises intriguing possibilities. What if a company could achieve positive returns with 50% more of its new-product introductions? What if marketing programs could generate 40% more customer inquiries? What if human resources could recruit 60% more of its highest-priority targets? What if twice as many workers were emotionally engaged in their jobs? Agile has brought these levels of improvement to IT. The opportunity in other parts of the company is substantial.
But a serious impediment exists. When we ask executives what they know about agile, the response is usually an uneasy smile and a quip such as “Just enough to be dangerous.” They may throw around agile-related terms (“sprints,” “time boxes”) and claim that their companies are becoming more and more nimble. But because they haven’t gone through training, they don’t really understand the approach. Consequently, they unwittingly continue to manage in ways that run counter to agile principles and practices, undermining the effectiveness of agile teams in units that report to them.
These executives launch countless initiatives with urgent deadlines rather than assign the highest priority to two or three. They spread themselves and their best people across too many projects. They schedule frequent meetings with members of agile teams, forcing them to skip working sessions or send substitutes. Many of them become overly involved in the work of individual teams. They talk more than listen. They promote marginal ideas that a team has previously considered and back-burnered. They routinely overturn team decisions and add review layers and controls to ensure that mistakes aren’t repeated. With the best of intentions, they erode the benefits that agile innovation can deliver.
Innovation is what agile is all about. Although the method is less useful in routine operations and processes, these days most companies operate in highly dynamic environments. They need not just new products and services but also innovation in functional processes, particularly given the rapid spread of new software tools. Companies that create an environment in which agile flourishes find that teams can churn out innovations faster in both those categories.
From our work advising and studying such companies, we have discerned six crucial practices that leaders should adopt if they want to capitalize on agile’s potential.
1. Learn How Agile Really Works
Some executives seem to associate agile with anarchy (everybody does what he or she wants to), whereas others take it to mean “doing what I say, only faster.” But agile is neither. (See the sidebar “Agile Values and Principles.”) It comes in several varieties, which have much in common but emphasize slightly different things. They include scrum, which emphasizes creative and adaptive teamwork in solving complex problems; lean development, which focuses on the continual elimination of waste; and kanban, which concentrates on reducing lead times and the amount of work in process. One of us (Jeff Sutherland) helped develop the scrum methodology and was inspired to do so in part by “The New New Product Development Game,” a 1986 HBR article coauthored by another of us (Hirotaka Takeuchi). Because scrum and its derivatives are employed at least five times as often as the other techniques, we will use its methodologies to illustrate agile practices.
The fundamentals of scrum are relatively simple. To tackle an opportunity, the organization forms and empowers a small team, usually three to nine people, most of whom are assigned full-time. The team is cross-functional and includes all the skills necessary to complete its tasks. It manages itself and is strictly accountable for every aspect of the work.
The team’s “initiative owner” (also known as a product owner) is ultimately responsible for delivering value to customers (including internal customers and future users) and to the business. The person in this role usually comes from a business function and divides his or her time between working with the team and coordinating with key stakeholders: customers, senior executives, and business managers. The initiative owner may use a technique such as design thinking or crowdsourcing to build a comprehensive “portfolio backlog” of promising opportunities. Then he or she continually and ruthlessly rank-orders that list according to the latest estimates of value to internal or external customers and to the company.
The initiative owner doesn’t tell the team who should do what or how long tasks will take. Rather, the team creates a simple road map and plans in detail only those activities that won’t change before execution. Its members break the highest-ranked tasks into small modules, decide how much work the team will take on and how to accomplish it, develop a clear definition of “done,” and then start building working versions of the product in short cycles (less than a month) known as sprints. A process facilitator (often a trained scrum master) guides the process. This person protects the team from distractions and helps it put its collective intelligence to work.
The process is transparent to everyone. Team members hold brief daily “stand-up” meetings to review progress and identify roadblocks. They resolve disagreements through experimentation and feedback rather than endless debates or appeals to authority. They test small working prototypes of part or all of the offering with a few customers for short periods of time. If customers get excited, a prototype may be released immediately, even if some senior executive isn’t a fan, or others think it needs more bells and whistles. The team then brainstorms ways to improve future cycles and prepares to attack the next top priority.
Agile Values and Principles
In 2001, 17 rebellious software developers (including Jeff Sutherland) met in Snowbird, Utah, to share ideas for improving traditional “waterfall” development, in which detailed requirements and execution plans are created up front and then passed sequentially from function to function. This approach worked fine in stable environments, but not when software markets began to change rapidly and unpredictably. In that scenario, product specifications were outdated by the time the software was delivered to customers, and developers felt oppressed by bureaucratic procedures.
The rebels proposed four new values for developing software, described principles to guide adherence to those values, and dubbed their call to arms “The Agile Manifesto.” To this day, development frameworks that follow these values and principles are known as agile techniques. Here is an adapted version of the manifesto: People Over Processes and Tools
Projects should be built around motivated individuals who are given the support they need and trusted to get the job done. Teams should abandon the assembly-line mentality in favor of a fun, creative environment for problem solving, and should maintain a sustainable pace. Employees should talk face-to-face and suggest ways to improve their work environment. Management should remove impediments to easier, more fruitful collaboration.Working Prototypes Over Excessive Documentation
Innovators who can see their results in real market conditions will learn faster, be happier, stay longer, and do more-valuable work. Teams should experiment on small parts of the product with a few customers for short periods, and if customers like them, keep them. If customers don’t like them, teams should figure out fixes or move on to the next thing. Team members should resolve arguments with experiments rather than endless debates or appeals to authority.Respond to Change Rather Than Follow a Plan
Most detailed predictions and plans of conventional project management are a waste of time and money. Although teams should create a vision and plan, they should plan only those tasks that won’t have changed by the time they get to them. And people should be happy to learn things that alter their direction, even late in the development process. That will put them closer to the customer and make for better results.Customer Collaboration Over Rigid Contracts
Time to market and cost are paramount, and specifications should evolve throughout the project, because customers can seldom predict what they will actually want. Rapid prototyping, frequent market tests, and constant collaboration keep work focused on what they will ultimately value. Read more
Compared with traditional management approaches, agile offers a number of major benefits, all of which have been studied and documented. It increases team productivity and employee satisfaction. It minimizes the waste inherent in redundant meetings, repetitive planning, excessive documentation, quality defects, and low-value product features. By improving visibility and continually adapting to customers’ changing priorities, agile improves customer engagement and satisfaction, brings the most valuable products and features to market faster and more predictably, and reduces risk. By engaging team members from multiple disciplines as collaborative peers, it broadens organizational experience and builds mutual trust and respect. Finally, by dramatically reducing the time squandered on micromanaging functional projects, it allows senior managers to devote themselves more fully to higher-value work that only they can do: creating and adjusting the corporate vision; prioritizing strategic initiatives; simplifying and focusing work; assigning the right people to tasks; increasing cross-functional collaboration; and removing impediments to progress.
2. Understand Where Agile Does or Does Not Work
Agile is not a panacea. It is most effective and easiest to implement under conditions commonly found in software innovation: The problem to be solved is complex; solutions are initially unknown, and product requirements will most likely change; the work can be modularized; close collaboration with end users (and rapid feedback from them) is feasible; and creative teams will typically outperform command-and-control groups.
In our experience, these conditions exist for many product development functions, marketing projects, strategic-planning activities, supply-chain challenges, and resource allocation decisions. They are less common in routine operations such as plant maintenance, purchasing, sales calls, and accounting. And because agile requires training, behaviorial change, and often new information technologies, executives must decide whether the anticipated payoffs will justify the effort and expense of a transition.
The Right Conditions for Agile
Agile innovation also depends on having a cadre of eager participants. One of its core principles is “Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.” When the majority of a company, a function, or a team chooses to adopt agile methodologies, leaders may need to press the holdouts to follow suit or even replace them. But it’s better to enlist passionate volunteers than to coerce resisters.
OpenView Venture Partners, a firm that has invested in about 30 companies, took this path. Having learned about agile from some of the companies in its portfolio, Scott Maxwell, the firm’s founder, began using its methodologies at the firm itself. He found that they fit some activities more easily than others. Agile worked well for strategic planning and marketing, for instance, where complex problems can often be broken into modules and cracked by creative multidisciplinary teams. That wasn’t the case for selling: Any sales call can change a representative’s to-do list on the spot, and it would be too complicated and time-consuming to reassemble the sales team, change the portfolio backlog, and reassign accounts every hour.
Agile innovation depends on having a cadre of eager participants.
Maxwell provided the companies in OpenView’s portfolio with training in agile principles and practices and let them decide whether to adopt the approach. Some of them immediately loved the idea of implementing it; others had different priorities and decided to hold off. Intronis was one fan. Its marketing unit at the time relied on an annual plan that focused primarily on trade shows. Its sales department complained that marketing was too conservative and not delivering results. So the company hired Richard Delahaye, a web developer turned marketer, to implement agile. Under his guidance the marketing team learned, for example, how to produce a topical webinar in a few days rather than several weeks. (A swiftly prepared session on CryptoLocker malware attracted 600 registrants—still a company record.) Team members today continue to create calendars and budgets for the digital marketing unit, but with far less line-item detail and greater flexibility for serendipitous developments. The sales team is much happier.
3. Start Small and Let the Word Spread
Large companies typically launch change programs as massive efforts. But the most successful introductions of agile usually start small. They often begin in IT, where software developers are likely to be familiar with the principles. Then agile might spread to another function, with the original practitioners acting as coaches. Each success seems to create a group of passionate evangelists who can hardly wait to tell others in the organization how well agile works.
The adoption and expansion of agile at John Deere, the farm equipment company, provides an example. George Tome, a software engineer who had become a project manager within Deere’s corporate IT group, began applying agile principles in 2004 on a low-key basis. Gradually, over several years, software development units in other parts of Deere began using them as well. This growing interest made it easier to introduce the methodology to the company’s business development and marketing organizations.
In 2012 Tome was working as a manager in the Enterprise Advanced Marketing unit of the R&D group responsible for discovering technologies that could revolutionize Deere’s offerings. Jason Brantley, the unit head, was concerned that traditional project management techniques were slowing innovation, and the two men decided to see whether agile could speed things up. Tome invited two other unit managers to agile training classes. But all the terminology and examples came from software, and to one of the managers, who had no software background, they sounded like gibberish. Tome realized that others would react the same way, so he tracked down an agile coach who knew how to work with people without a software background. In the past few years he and the coach have trained teams in all five of the R&D group’s centers. Tome also began publishing weekly one-page articles about agile principles and practices, which were e-mailed to anyone interested and later posted on Deere’s Yammer site. Hundreds of Deere employees joined the discussion group. “I wanted to develop a knowledge base about agile that was specific to Deere so that anyone within the organization could understand it,” Tome says. “This would lay the foundation for moving agile into any part of the company.”
Using agile techniques, Enterprise Advanced Marketing has significantly compressed innovation project cycle times—in some cases by more than 75%. One example is the development in about eight months of a working prototype of a new “machine form” that Deere has not yet disclosed. “If everything went perfectly in a traditional process,” Brantley says, “it would be a year and a half at best, and it could be as much as two and a half or three years.” Agile generated other improvements as well. Team engagement and happiness in the unit quickly shot from the bottom third of companywide scores to the top third. Quality improved. Velocity (as measured by the amount of work accomplished in each sprint) increased, on average, by more than 200%; some teams achieved an increase of more than 400%, and one team soared 800%.
Success like this attracts attention. Today, according to Tome, in almost every area at John Deere someone is either starting to use agile or thinking about how it could be used.
4. Allow “Master” Teams to Customize Their Practices
Japanese martial arts students, especially those studying aikido, often learn a process called shu-ha-ri. In the shu state they study proven disciplines. Once they’ve mastered those, they enter the ha state, where they branch out and begin to modify traditional forms. Eventually they advance to ri, where they have so thoroughly absorbed the laws and principles that they are free to improvise as they choose.
Mastering agile innovation is similar. Before beginning to modify or customize agile, a person or team will benefit from practicing the widely used methodologies that have delivered success in thousands of companies. For instance, it’s wise to avoid beginning with part-time assignment to teams or with rotating membership. Empirical data shows that stable teams are 60% more productive and 60% more responsive to customer input than teams that rotate members.
Over time, experienced practitioners should be permitted to customize agile practices. For example, one principle holds that teams should keep their progress and impediments constantly visible. Originally, the most popular way of doing this was by manually advancing colored sticky notes from the “to-do” column to “doing” to “done” on large whiteboards (known as kanban boards). Many teams are still devoted to this practice and enjoy having nonmembers visit their team rooms to view and discuss progress. But others are turning to software programs and computer screens to minimize input time and allow the information to be shared simultaneously in multiple locations.
A key principle guides this type of improvisation: If a team wants to modify particular practices, it should experiment and track the results to make sure that the changes are improving rather than reducing customer satisfaction, work velocity, and team morale.
Spotify, the music-streaming company, exemplifies an experienced adapter. Founded in 2006, the company was agile from birth, and its entire business model, from product development to marketing and general management, is geared to deliver better customer experiences through agile innovation. But senior leaders no longer dictate specific practices; on the contrary, they encourage experimentation and flexibility as long as changes are consistent with agile principles and can be shown to improve outcomes. As a result, practices vary across the company’s 70 “squads” (Spotify’s name for agile innovation teams) and its “chapters” (the company term for functional competencies such as user interface development and quality testing). Although nearly every squad consists of a small cross-functional team and uses some form of visual progress tracking, ranked priorities, adaptive planning, and brainstorming sessions on how to improve the work process, many teams omit the “burndown” charts (which show work performed and work remaining) that are a common feature of agile teams. Nor do they always measure velocity, keep progress reports, or employ the same techniques for estimating the time required for a given task. These squads have tested their modifications and found that they improve results.
5. Practice Agile at the Top
Some C-suite activities are not suited to agile methodologies. (Routine and predictable tasks—such as performance assessments, press interviews, and visits to plants, customers, and suppliers—fall into this category.) But many, and arguably the most important, are. They include strategy development and resource allocation, cultivating breakthrough innovations, and improving organizational collaboration. Senior executives who come together as an agile team and learn to apply the discipline to these activities achieve far-reaching benefits. Their own productivity and morale improve. They speak the language of the teams they are empowering. They experience common challenges and learn how to overcome them. They recognize and stop behaviors that impede agile teams. They learn to simplify and focus work. Results improve, increasing confidence and engagement throughout the organization.
A number of companies have reallocated 25% or more of selected leaders’ time from functional silos to agile leadership teams. These teams rank-order enterprisewide portfolio backlogs, establish and coordinate agile teams elsewhere in the organization to address the highest priorities, and systematically eliminate barriers to their success. Here are three examples of C-suites that took up agile:
1. Catching up with the troops.
Systematic, a 525-employee software company, began applying agile methodologies in 2005. As they spread to all its software development teams, Michael Holm, the company’s CEO and cofounder, began to worry that his leadership team was hindering progress. “I had this feeling that I was saying, ‘Follow me—I’m just behind you,’” he told us. “The development teams were using scrum and were doing things differently, while the management team was stuck doing things the same old-fashioned way”—moving too slowly and relying on too many written reports that always seemed out-of-date. So in 2010 Holm decided to run his nine-member executive group as an agile team.
The team reprioritized management activities, eliminating more than half of recurring reports and converting others to real-time systems while increasing attention to business-critical items such as sales proposals and customer satisfaction. The group started by meeting every Monday for an hour or two but found the pace of decision making too slow. So it began having daily 20-minute stand-ups at 8:40 am to discuss what members had done the day before, what they would do that day, and where they needed help. More recently the senior team began to use physical boards to track its own actions and the improvements coming from the business units. Other functions, including HR, legal, finance, and sales, now operate in much the same way.
2. Speeding a corporate transition.
In 2015 General Electric rebranded itself as a “digital industrial company,” with a focus on digitally enabled products. Part of the transformation involved creating GE Digital, an organizational unit that includes all 20,000-plus of the company’s software-related employees. Brad Surak, who began his career as a software engineer and is now GE Digital’s COO, was intimately familiar with agile. He piloted scrum with the leadership team responsible for developing industrial internet applications and then, more recently, began applying it to the new unit’s management processes, such as operating reviews. Surak is the initiative owner, and an engineering executive is the scrum master. Together they have prioritized backlog items for the executive team to address, including simplifying the administrative process that teams follow to acquire hardware and solving knotty pricing issues for products requiring input from multiple GE businesses.
The scrum team members run two-week sprints and conduct stand-up meetings three times a week. They chart their progress on a board in an open conference room where any employee can see it. Surak says, “It takes the mystery out of what executives do every day. Our people want to know if we are in tune with what they care about as employees.” The team collects employee happiness surveys, conducts root cause analysis on the impediments to working more effectively, and reports back to people throughout the organization, saying (in effect), “We heard you. Here is how we will improve things.” Surak believes that this shows the organization that “executives work in the same ways as engineers,” increasing employee motivation and commitment to agile practices.
3. Aligning departments and functions on a common vision.
Erik Martella, the vice president and general manager of Mission Bell Winery, a production facility of Constellation Brands, introduced agile and helped it spread throughout the organization. Leaders of each department served as initiative owners on the various agile teams within their departments. Those individual teams achieved impressive results, but Martella worried that their time was being spread too thin and that department and enterprise priorities weren’t always aligned. He decided to pull department leaders into an executive agile team focused on the enterprise initiatives that held the greatest value and the greatest opportunity for cross-functional collaboration, such as increasing process flows through the warehouse.
The team is responsible for building and continually refining the backlog of enterprise priorities, ensuring that agile teams are working on the right problems and have sufficient resources. Team members also protect the organization from pet projects that don’t deserve high priority. For instance, shortly after Martella started implementing agile, he received an e-mail from a superior in Constellation’s corporate office suggesting that the winery explore a personal passion of the sender. Previously, Martella might have responded, “OK, we’ll jump right on it.” Instead, he replied that the winery was following agile principles: The idea would be added to the list of potential opportunities and prioritized. As it happened, the executive liked the approach—and when he was informed that his suggestion had been assigned a low priority, he readily accepted the decision.
Scrum “takes the mystery out of what executives do every day.”
Working on agile teams can also help prepare functional managers—who rarely break out of their silos in today’s overspecialized organizations—for general management roles. It exposes them to people in other disciplines, teaches collaborative practices, and underscores the importance of working closely with customers—all essential for future leaders.
6. Destroy the Barriers to Agile Behaviors
Research by Scrum Alliance, an independent nonprofit with 400,000-plus members, has found that more than 70% of agile practitioners report tension between their teams and the rest of the organization. Little wonder: They are following different road maps and moving at different speeds.
Here’s a telling example: A large financial services company we examined launched a pilot to build its next mobile app using agile methodologies. Of course, the first step was to assemble a team. That required a budget request to authorize and fund the project. The request went into the batch of submissions vying for approval in the next annual planning process. After months of reviews, the company finally approved funding. The pilot produced an effective app that customers praised, and the team was proud of its work. But before the app was released, it had to pass vulnerability testing in a traditional “waterfall” process (a protracted sequence in which the computer code is tested for documentation, functionality, efficiency, and standardization), and the queue for the process was long. Then the app had to be integrated into core IT systems—which involved another waterfall process with a six-to-nine-month logjam. In the end, the total time to release improved very little.
Here are some techniques for destroying such barriers to agile:
Get everyone on the same page.
Individual teams focusing on small parts of large, complex problems need to see, and work from, the same list of enterprise priorities—even if not all the teams responsible for those priorities are using agile processes. If a new mobile app is the top priority for software development, it must also be the top priority for budgeting, vulnerability testing, and software integration. Otherwise, agile innovations will struggle in implementation. This is a key responsibility of an executive team that itself practices agile.
Don’t change structures right away; change roles instead.
Many executives assume that creating more cross-functional teams will necessitate major changes in organizational structure. That is rarely true. Highly empowered cross-functional teams do, by definition, need some form of matrix management, but that requires primarily that different disciplines learn how to work together simultaneously rather than separately and sequentially.
Name only one boss for each decision.
People can have multiple bosses, but decisions cannot. In an agile operating model it must be crystal clear who is responsible for commissioning a cross-functional team, selecting and replacing team members, appointing the team leader, and approving the team’s decisions. An agile leadership team often authorizes a senior executive to identify the critical issues, design processes for addressing them, and appoint a single owner for each innovation initiative. Other senior leaders must avoid second-guessing or overturning the owner’s decisions. It’s fine to provide guidance and assistance, but if you don’t like the results, change the initiative owner—don’t incapacitate him or her.
Focus on teams, not individuals.
Studies by the MIT Center for Collective Intelligence and others show that although the intelligence of individuals affects team performance, the team’s collective intelligence is even more important. It’s also far easier to change. Agile teams use process facilitators to continually improve their collective intelligence—for example, by clarifying roles, teaching conflict resolution techniques, and ensuring that team members contribute equally. Shifting metrics from output and utilization rates (how busy people are) to business outcomes and team happiness (how valuable and engaged people are) also helps, as do recognition and reward systems that weight team results higher than individual efforts.
Lead with questions, not orders.
General George S. Patton Jr. famously advised leaders never to tell people how to do things: “Tell them what to do, and they will surprise you with their ingenuity.” Rather than give orders, leaders in agile organizations learn to guide with questions, such as “What do you recommend?” and “How could we test that?” This management style helps functional experts grow into general managers, and it helps enterprise strategists and organizations evolve from silos battling for power and resources into collaborative cross-functional teams.
Agile innovation has revolutionized the software industry, which has arguably undergone more rapid and profound change than any other area of business over the past 30 years. Now it is poised to transform nearly every other function in every industry. At this point, the greatest impediment is not the need for better methodologies, empirical evidence of significant benefits, or proof that agile can work outside IT. It is the behavior of executives. Those who learn to lead agile’s extension into a broader range of business activities will accelerate profitable growth.
Further Resources
Agile Alliance: For guides to agile practices, links to “The Agile Manifesto,” and training videos
Scrum Alliance: For a “Scrum Guide,” conference presentations and videos, and the “State of Scrum” research report
ScrumLab Open: For training presentations, videos, webinars, and published papers
Annual State of Agile Survey: For key statistics such as usage rates, customer benefits, barriers to adoption and success, and specific practices used