The Rise Of Shadow IT

Jul 17, 2012 (07:07 AM EDT)

Read the Original Article at

Part One: IT Is from Mars, Business Is from Venus

We've moved from a world of scarce IT--where organizations guard their technology--to one of abundance, where technology permeates every part of our daily lives.

But the enterprise IT team is still acting like IT is a scarce, precious resource. They're charged with protecting and controlling its use, reducing risk; while the world of clouds, startups, and innovation focuses on its abundance, maximizing its rewards.

Once, IT was protected. And with good reason: computers cost millions, broke easily, and represented a vast barrier to entry. Companies competed based on their ability to corral capital, infrastructure, and labor--to control the means of production.

[ For more cloud computing analysis, see Top 12 Cloud Trends Of 2012. ]

Today, however, industry after industry is being disrupted by startups that see these once-daunting barriers to entry as crumbling castle walls. The line of business wants to innovate, and central IT is facing an internal revolt: Shadow IT. We can track the rise of shadow IT to three things: the personal computer; data carriers; and software-as-a-service.

When the first home spreadsheet came out, early adopters found ways to use it. But unlike central IT, which could be tightly controlled, clients had little to stop employees from going rogue. Visicalc was the first assault on the walls of central IT; the rise of the Web only made things worse. Despite every attempt to lock down desktops, employees found ways to use their own tools.

A second flank of the assault happened when salespeople, eager for an edge on their quarterly targets, embraced two-way pagers. The Blackberry was a rogue device, but the political capital of its users made it impossible for IT to refuse them outright.

But it was software-as-a-service that really hammered the nails into the coffin of centralization. Not only were SaaS sites pay-as-you-go, but they were trivially easy to deploy and configure. Departments started circumventing IT for everything from CRM to web analytics, performance monitoring, expense management, invoicing, and more.

Every organization has an immune system. Enterprise IT has been particularly well entrenched, because to them, change is bad. Change is the leading cause of outages and downtime, so it is to be avoided.

But to the line of business, change is good. Without change, things stagnate, and competitors get the upper hand. The tension between innovation and operation was almost unbearable. And the rise of cloud computing was the crack that broke open the dam.

Part Two: Therapy

Now I'm going to play therapist. Why does the line of business want change so much it's willing to create its own shadow IT organization? Is it because it hates enterprise IT? Does it think it can do a better job? Those would be pathological reasons. If we assume that it's rational, then what it should want is to raise profits and lower costs--to turn resources into value. That's what everyone should want.

There are two possibilities. Either it's going rogue because it's misinformed, or it's going rogue because it's justified.

Is it misinformed? There's no doubt that public clouds have privacy, governance, and lock-in concerns that the line of business might not be aware of. There's also little doubt that for a predictable, stable workload, the raw cost of on-demand computing is higher than owning your own machines because you're paying someone else's profit, and paying for variance--this is simply a function of Just-In-Time economics.

Is it right? It takes far too long to spin up internal resources--often longer than the lifetime of the application--and today's climate of experimentation exacerbates the issue.

But there's a much more fundamental reason this couple isn't talking: They're not using the same language.

The line of business thinks top-down, and it wants to go down as little as possible. It starts by saying, "I want a website for this promotion," and goes only as deep as it needs to.

On the other hand, the enterprise IT department thinks bottoms up, and it wants to go up as little as possible. It starts by saying, "You'll need three 4-core servers and 2TB of data, running Linux, for a month," and only goes up as high as it needs to.

When enterprise IT says it's building a cloud, it's using a bottom-up definition: machines available when you need them, paid for based on the technical resources (RAM, compute cycles, storage, bytes sent) consumed. Think virtualization, plus.

By contrast, when the line of business says it's using a cloud, it's describing a top-down definition: a functional piece of software that just works, paid for based on consumption, seats, or some other business metric. Think Salesforce, minus.

And this, right here, is the reason they want a divorce.

I call this the service gap.

Part Three: What's a CIO to Do?

So what's an enterprise IT professional to do? As it turns out, plenty. Good CIOs manage processes. But great CIOs ask the business to bring them problems, and then they try to solve those problems using technology. And in a connected, data-driven world, there's no shortage of problems to tackle. They just require different solutions.

Take a look at a public cloud provider. They have dozens of services--from compute, to storage, to message busses, to human APIs, to mailing systems, to payment.

Which restaurant would you rather eat at?

Resisting the variety they crave is deadly. But building a set of services they can embrace is great. Because when you build services, you don't give them access to the underlying components. You keep control of those.

And that's good for everything from governance, to licensing cost, to operational overhead.

Here's a concrete example: Storage. Most developers say, "I want a database." What they really want is a service they can query; today, that request is usually translated by IT (with its device-centric hearing) into, "I want a database server."

The architecture of a database depends a lot on how it's used. Cassandra is a kind of data store that supports fast injection of new data, for example. A traditional RDBMS is great for joining related tables together. Other architectures and tools work well for other applications--fast concurrent search results, large object storage, reliable, geographically redundant storage, and so on.

If, rather than spending a week deliberating and issuing a server, the CIO's team stands up a service, several things happen. First, the application performance will be better, because it'll be tailored to the underlying infrastructure. Second, the licensing will be easier, because there'll be a single set of underlying tools and the variety happens at the API layer, not at the component layer. And third, governance and automation is easier to manage because of standardization.

But this doesn't happen automatically. It takes a real effort of will for a traditional, bottoms-up, change-adverse IT team to see itself as the creator of services.

In other words, you need to switch from a culture of scarcity (where IT is a precious resource to be protected) to a culture of abundance (where it's an open tool for innovation) by building a rich ecosystem of services atop infrastructure you control--in both public and private cloud environments.

Rather than resisting change, IT executives need to embrace it—and appeal to basic human motivations with a little hustle, a bit of seduction, a dash of arm-twisting, and a sprinkling of raw terror. The rise of shadow IT is a topic we'll discuss during an upcoming Cloud Connect webinar on July 26th.

Alistair Croll, founder of analyst firm Bitcurrent, is conference chair of the Cloud Connect events.

Cloud Connect is expanding to the Windy City. Join 1,200+ IT professionals at Cloud Connect Chicago, where you will learn how to leverage new cloud technology solutions to increase productivity and improve your business agility. Join us in Chicago, Sept. 10-13. Register today!