Why innovation fails

It sometimes feels like business dances to the drumbeat of innovation — but the vast majority of innovation projects fail. Why is that?

Innovation. Innovation. Innovation.

Over the last decade or so, innovation has become a corporate watchword — no, a corporate obsession. CEOs and press releases don’t crow about the efficiency of their operation, or its revenue. Instead they bang on about the innovation at work. Everything is “innovative”, in a pale imitation of the late Steve Jobs at his reality distorting best — it’s the one virtue that all businesses have to have.

Why then, do most innovations fail? Some estimates put the failure rate at something like 90% of all innovation projects. Those are terrible odds. Why do people keep playing this game? And, more to the point, how can we improve those odds for our own innovation projects?

Understanding Innovation

First of all, we need to understand innovation. It often isn’t what it seems; innovation is not just "new stuff". As Arthur Levine wrote in his 1980 book Why Innovation Fails:

Innovation combines the elements of reform and change; reform implying new and change implying different. Innovation can operationally be defined as any departure from the traditional practices of an organisation. As a result, the element of newness inherent in innovation is a relative phenomenon — what is new in one place is old in the next. The same might be said of time. Much that we call innovation is in fact renovation.

Innovation isn’t equally distributed, but a relative concept. Old ideas might have new relevance when technology advances to the point they become feasible. More than one technology has failed through being just too early. Look at the iPad: Microsoft was there, in the tablet market, nearly a decade before Apple. But it was too early.

Truly disruptive innovations are few and far between. And that’s a good thing - the consequences of them take decades to work through. We’re still wrestling with the double-whammy of the internet and the smartphone right now.

But incremental innovation is important to all businesses, especially in the wake of the disruptive innovations and their aftershocks. This form of innovation isn't about wild flights of fancy. It's about process and hard work.

What, No Structure?

In that context, you need to examine the reality of your idea. Just how good is it really? Fortune wrote about the fundamental tests that innovation needs to go through back in 2014:

It can get rough, and for a down-to-earth reason. “Ideas are treated not as precious pearls to be polished, but as sparks born of friction”—because “exposing fledgling innovation ideas to the tough love of tough questions … ensures those ideas can survive in the real world of real companies placing real bets with real money.”

Not all new ideas are good. There’s a certain inherent excitement in a new idea — but all too often it starts to lose its lustre when exposed to the harsh reality of implementation. The “what” can struggle when it clashes with the “how”. One of the reasons innovation projects fail is that not enough hard questions are asked in the early stages of a project. There are some ideas that don't deserve to survive to implementation.

In short: ideas are abundant. Good, practical ideas are significantly less so.

The sooner you can sort your bad ideas from your good ones, the more time and energy you have to invest in the ones that survive. And to make them work they need a form, a process: a structure, in fact. That’s the journey that takes you from an idea to an innovation.

It may all start with a burst of ideas on PostIt notes — but it’s the process that takes those ideas, sifts them for the gems, and then tests them at every stage of development that makes innovation truly powerful.

Dodging the corporate immune system

Of course, there’s an inverse danger. (There’s always an inverse danger.) And that danger is allowing the existing mechanisms of the business to stifle the idea. Hoorah - you just came up with a great idea! Now, work out a long business case for your idea, push it through five committees who have no interest in seeing it succeed, because its doesn’t benefit them personally or politically, and what comes out the other side will look little like what you started with.

This you could describe as the corporate immune system. Most traditionally-structured businesses are built to deliver existing, known processes with ever-greater efficiency. An innovation is all too often a disruption to that process, and the company itself, with its policing departments — operations, HR, IT — will attempt to kill it, because it doesn’t fit within their rules. That’s their job. They exist to protect the established, trusted processes.

One of the major areas of exploration right now is new ways of structuring companies that allow innovations to be rigorously explored, without the policing effect strangling them at birth. Killing off an innovative idea is not inherently a bad thing — not all ideas are good — but doing so before it is properly explored most certainly is.

So, why does innovation fail?

Innovation fails because you don’t test it systematically, and because you don’t protect the innovation from habitual challenges designed to protect the status quo.

Reconciling this requires a business that can manage parallel worlds - one of existing business operating as before, and one of exploration and testing: establishing the innovations that will, in time, become the new status quo.

Managing these Parallelwelten is beyond most businesses; and that’s why they fail. Doing both is the challenge for any business in the digital age.


Photo by Nathan Dumlao on Unsplash