Over the past thirty years, we have created this field of business analysis around a few key foundational points:

1. That requirements development is a complex-enough area to be worthy of planning
2. That requirements documentation produces value to its stakeholders and, by extension, the enterprise
3. That business and market change is slow enough – and requirements static enough – for a developed product to still be relevant by the end of the project

These foundational points have been instrumental in solving key project management problems. Project success rates improved as requirements development processes matured. We helped organizations deliver more valuable projects by keeping a tighter grip on scope and focusing the project scope around prioritized business requirements. And by building relationships with clients and project sponsors, we gave these executives a view and interaction point with technology divisions that is stronger now than ever.

However, changes in customer demand and big technology vectors (e.g. web, mobile, social, cloud, DevOps; see Illustration 1) have changed the dynamics and economics of software and product development. The resulting technology organizational changes have necessarily made our enterprises more agile and lean. The natural outgrowth of this will be changes to how business analysis gets done.

Indeed, these changes to our ecosystem are creating an entirely new approach to business analysis, centered on collaboration, a shifting of emphasis from artifacts to actual product, a more direct focus on enterprise value, a lessening of the importance of project-level analysis in favor of enterprise analysis, and a corresponding increase in the overall value of the Business Analyst to the enterprise.

Changes to any ecosystem invariably benefit some individuals and harm others. We have seen winners and losers as Agile, Lean, and other factors have been introduced and have grown to maturity. Similarly, some organizations will thrive in the new business analysis ecosystem, and others will not.

To understand how these changes affect us, we have to look at the history of how we got here.

The Evolution of Business Analysis

Pre-History

Like all of evolution, there was no clear starting point to business analysis as a profession (or the role of the Business Analyst, for that matter). The role started to emerge in the 1980s and was clearly in existence by the 1990s. However, it was really during the latter that business analysis as an area distinct from software engineering arose.

1995 was a pivotal year for the profession as the Standish Group published their CHAOS Report. Although this research report has been used well beyond its reasonable limits, it did offer a point-in-time illustration of deficiencies in 90s-era software project management. Among business analysis professionals, its most salient point was that lack of user involvement and incomplete or changing requirements specifications were the largest contributing factors to projects being challenged or fail-ing. This rightly caused us to ask, “How can we make requirements analysis work better, so that our projects will be more reliably delivered?” 

We continued to ask that question for the following ten years, more or less productively. We found and published best practices (e.g. interviewing customers, observing users). We adapted tools from other fields (e.g. UML from software engineering, business process modeling from industry). And we made some attempts, none fully realized, to introduce common business analysis idioms (e.g. requirement patterns, conceptually borrowed from software design patterns). By the early 2000s, we had a pretty good handle on what business analysis was and how to do it effectively.

Solidification

At the same time, however, we had an existential crisis. No one really knew what a Business Analyst was. There was little consistency across industries. Although we knew what we ourselves did, and we could get along well when happening across a BA in another industry or geography, we longed for a better definition. We wanted to be able to explain our careers to people in fifty words or less. We envied lawyers, doctors, accountants, and programmers for their ability to instantly kill a “What do you do for a living?” conversation with a simple job title response.

Accordingly, the International Institute of Business Analysis (IIBA) was founded in 2003, ostensibly to solidify business analysis as a profession. They quickly established chapters in the world’s major metropolitan areas and encouraged an exchange of ideas, best practices, and empathy. They introduced certifications for business analysis practitioners. But their biggest impact has been the development of the Guide to the Business Analysis Body of Knowledge (BABOK). BABOK served as a framework upon which we could organize the field. Lean on full-time staff, IIBA encouraged senior business analysis professionals to come together, propose best practices, conduct peer reviews, and publish. With the stroke of a pen, IIBA gave us a definition that would stick (with minor alterations) for years to come: “Business analysis is the set of tasks, knowledge, and techniques required to identify business needs and determine solutions to business problems” (BABOK, version 1.6). By 2005, we knew who we were, we knew what we did, and we were damn proud of it. 

Agile and Lean

And then Agile came along. Agile had of course been around for quite a while, the Agile Manifesto being signed in 2001. Agile itself was an outgrowth of Extreme Programming (XP), Adaptive Software Development, and other approaches developed in the 1990s. 

Agile called for people to be the driving force behind software development, not a plan, not a Statement of Work, not a specification. Agilists valued “individuals and interactions over processes and tools” and espoused other progressive organizational values that so many of us had felt for so long. It was a welcome respite from the heavy methodologies we were burdened by. 

But Agile also presented two major challenges to business analysis orthodoxy:

1. It called for direct collaboration between the customer (typically represented by a “product owner”) and the development team. This was a problem for those of us who represented our-profession as serving as a “bridge” between business and IT stakeholders
2. It literally devalued the role of documentation in favor of “working software” (or whatever the equivalent product was for your area). This was a problem for those of us whose progress was measured largely by the number of requirements documents churned out in support of our initiatives

Agile made short work of each of the foundational points of our new profession, mentioned in the first paragraph of this paper. We were sunk.

Thus, our second great existential crisis arose, and it was far more serious. Business analysis would slowly disappear as developers and customers collaborated, edging us out of the process. There were two notable reactions to this. Some predicted that Agile would forever remain an alternative approach to running some projects. They pointed out that no great construction projects were done using agile methods. This is the “would you cross a bridge made using Agile?” argument. The counterargument was of course that business analysts were rarely if ever involved in constructing bridges, and that Agile went on to be an effective methodology on that subset of projects that BAs did get involved with. Indeed, by 2009, waterfall-oriented project work had dropped to a small fraction of all development processes, with agile approaches taking the lead.

Others sought to examine the value that business analysis plays in the organizational ecosystem. Independent thought leaders contributed, but IIBA again played a strong role in championing the growth of (non-project-based) enterprise business analysis, and, with time, promoted a focus on non-documentary agile business analysis methods – most notably through publishing its Agile Extension to the BABOK Guide. Agile was clearly here to stay, and it was imperative that we adapt to it.

We also started to see the rising dynamic of Lean entering the broader business arena. Lean had long been a mainstay of manufacturing, but its concept of maximizing customer value through reduction of waste was increasingly necessary as the world plunged into economic crisis in 2009. We had to do more with less and the reduction in resources combined with increased consumer expectations had forced us to be more inventive in focusing on our most important priorities.

Lean encourages Business Analysts to change our frame of reference. In the 1990s, we asked, “How can we make requirements analysis work better, so that our projects will be more reliably delivered?” Now we ask, “How can we do as little work as possible to deliver the tiniest product customers will value?”

In retrospect, it is somewhat surprising that Agile and Lean fit each other so well. Agile started out as a grass-roots, individual contributor-led approach to making customers happier and teams more sustainable. Lean, on the other hand, arose as a management-led necessity driven by larger-scale economic factors. That they ended up in similar places is perhaps the most compelling argument that flexible product development is the approach of the future.

Today

We now find business analysis at a crossroads. 

Business transformation has become such a ubiquitous part of corporate life that it has become business as usual. Indeed, there is no longer a meaningful business orthodoxy that we can point to as being transformable. Business models arise and disappear overnight, disrupted by new business models. As blue chip firms with hundreds of thousands of employees become “lean startups” and government becomes agile, the pace of change has become effectively asymptotic.

We now face an ecosystem much more challenging than anything we ever dealt with as Agile went mainstream. We are getting to a point where Business Analysts must take on an entirely new form if we are to survive and our enterprises are to thrive in this current environment.

The Future Business Analyst

Given these trends, and the rise of Agile and Lean, it will come as no surprise that certain organizations – and Business Analysts within those organizations – will be impacted. Some Business Analysts will adapt to the new environment and thrive; others less so.  

Winner: The Agile Business Analyst

The future Business Analyst will be agile by necessity.

Most Business Analysts today are focused on project requirements, and requirements absorb most of the BA’s time. This has to change, because companies need BAs to drive more valuable higher-level analysis work, and the time investment of requirement documentation is crowding out higher priority work.

BAs will continue to play a role in project requirements. This will not go away. They will continue to work on user stories and acceptance criteria and backlog management. But as organizations become more agile, BAs will be able to take advantage of the lessened need for documented requirements to save time. They’ll use this time to focus on questions like the following:

• Do we have the best product strategy?
• How does it need to change?
• Is the product we’re building in line with our product strategy? Our corporate strategy?

Although these questions will not be the sole domain of the Business Analyst – indeed, primary ownership must remain with corporate and product management – the critical analytical capabilities of the Business Analyst will be a major boon to the enterprise.

Business Analysts who attempt to ignore the trend will find themselves at a disadvantage at best. They will slowly but assuredly lose relevance, as they are increasingly held to a higher standard (and fall short). 

Winner: The Lean Business Analyst

Over the last five to ten years, we’ve seen an expansion of lean manufacturing principles – topics like waste reduction and customer value – into non-manufacturing areas. Most prominent, possibly, is a focus on entrepreneurship kick-started by Eric Ries’ The Lean Startup. Although targeted toward entrepreneurs, its lessons are being applied to the world’s largest companies and government organizations. NorwalkAberdeen predicts that this trend will continue, achieving mainstream adoption in the next five years. 

The principles will impact BAs in a couple areas:

Waste reduction. The primary area BAs will worry about is reducing wasted investment. Technology projects tend to be expensive, and little is done in practice to ensure that business goals will be achieved once the project is delivered. BAs must take on a renewed emphasis on making projects more likely to be successful and ensuring the invested capital is intelligently stewarded.

Experimentation. Just as marketers conduct market research to vet ideas prior to building products, we’ll see Business Analysts conducting experiments to validate that new products will be successful and new processes will be effective. All assumptions that we bake into projects today will be verified in the future through experimentation.
Business Analysts will be held accountable to this standard in the future, regardless of whether or not they choose to adhere to lean principles. In project reviews, the difference between lean and non-lean execution will be apparent to management. Those who run tight ships will thrive. Those who don’t will not.

Winner: The Strategy (FKA Enterprise) Business Analyst

The natural outcome of BAs being agile and lean will be a greater focus on companies themselves. BAs will be freed up from worrying about project-level details and will focus more on the value of the projects and making them successful. We’ll be thinking beyond our products and looking at how our products fit into the enterprise. We’ll ask questions like: How can my product be a boon to other product teams? How can we internally evangelize our products to those teams? Are there opportunities to synthesize products to meet new or existing customer needs? These questions, historically the domain of product management, will be a shared collaboration space for Product Managers and Business Analysts.

To date, most enterprise business analysis has not been done by business analysts. It has typically been done at a more senior level by corporate strategists or product management. The best candidates to produce high-quality enterprise analysis will have a combination of broad organizational focus, strong analytical capability, and superior interviewing and observational skills. 

We will have to take a more active approach in leading these efforts. CEOs and executives clearly see the need for this work to be done; that’s why they hire strategy consultants to come in and find opportunities. But executives tend to have a narrow view of business analysis. We are often thought of as “those IT people.” We need to push the boundaries by proposing these efforts to senior management.

Not all Business Analysts will make the leap to the enterprise analysis level. There will always be a need for detailed analysis work to be done, but those BAs will find themselves playing an increasingly junior role. 

The New Role

You don’t have to choose among the three winning strategies. You’ll optimize your position by pursuing all three at once. 

Agile Business Analysts, largely unshackled from a focus on documentation, will be able to focus their attention on value-added activities: product strategy-feature alignment, understanding the customer, and increasing the value of those features to the customer. 

Lean Business Analysts will enhance the likelihood of market success of their products and features by driving experimentation. Business Analysts are best suited to analyze where business requirements are based on untested assumptions. By designing early and low-investment experiments of assumptions, Business Analysts can determine what the market will truly value. Thus, BAs will ensure that product strategy is aligned with the realities of the marketplace as evidenced by those experiments. Implementing lean principles around experimentation will necessitate the Business Analyst being involved in product and project investment decisions.

While the practice of enterprise business analysis will remain largely the same, it will take on a much more important focus. As organizations adopt Agile and Lean on a broad scale, the natural effect will be Business Analysts focusing on the organization as a whole. 

The Business Analyst who synthesizes these three approaches to impacting the company’s bottom line will be formidable indeed.

The Business Analysts of the future are among us today.  They have a broader mindset, skill-set and scope.  By pursuing lifelong learning and taking an active role in their professional development, they will be able to navigate the challenges ahead.

Don Hussey is Managing Director of Advisory Services for NorwalkAberdeen. He is directly responsible for managing all business analysis activities of the firm, overseeing advisory engagements, training, and coaching.

Prior to co-founding NorwalkAberdeen, he was Senior Vice President and head of the Business Analysis Center of Excellence for Citi Private Bank.