Business Intelligence: 2012 to 2015

According to some interpretations, the end of the Mayan Calendar on 21 December, 2012 signifies the end of the world. Or, at least, a major transformation in consciousness. Whether this is true or not remains to be seen. However, if the past few years are any guide, we can certainly expect major, continuing change… this year and in those following.
And how business, and the world in general, has changed in the last decade! Dotcom boom to bust in the early years of the millennium; and the subsequent emergence of the web mega-fauna—Google, Facebook, eBay and more. The financial crisis of 2008 and the subsequent Euro sovereign debt crisis. The Arab Spring. General Motors, from world’s largest automobile maker for 77 years to bankruptcy in 2008 and re-emergence at the top spot in 2011. From three companies—Nokia, Research in Motion, and Motorola—dominating the smartphone market in 2006 to none of them among the leaders by last year. The US losing its AAA rating. The list goes on…
The Biz-Tech Ecosystem
If there is one single, common factor in all this tumultuous change, it is technology and its intersection with business and people. Allow me to introduce you to an important evolution in the business world. A new species of business is emerging. It has been vaguely visible for some time now. But its time has come. I call it the biz-tech ecosystem—a confluence of business needs and technology trends that overturn the fundamentals of many long-established practices in both business and IT. A new environment in which people live and work.
The initial technological signs can be seen as far back as the 1980s. The difference now, 30 years later, is that these seeds have matured and grown into a highly diverse and interdependent technology jungle that today’s business users dare to enter in significant numbers. And they are ready and able to do so; having grown up with computing and communications technology that, thirty years ago, would have been seen as magic by the vast majority of business people.
There are three key characteristics of the biz-tech ecosystem as it has emerged in the past few years:
1. Interdependence: Business and technology are each driven, one by the other, in a tight loop. New technology enables new business possibilities; new business opportunities drive advances in technology
2. Reintegration: The silos within and across both business and IT have grown increasingly uncomfortable to maintain; they deliver inefficiencies, miscommunications and errors. The cracks can no longer be hidden from Web-savvy customers; coherence is becoming mandatory
3. Cross-over: Business people need the skill to understand sufficient technology to envision how new advances could be used to recreate the business. Similarly, IT people need the business acumen to see how business needs can be satisfied in new ways by emerging technology
In many ways, these characteristics are diametrically opposed to the way business and IT have operated for five decades already—business determines their requirements, hands them over to IT and waits for an application to be delivered, determines that it was incomplete or incorrect and goes around the loop again. That era is coming to a close and, with it, our entire approach to business intelligence is set to change. In the coming few years, successful businesses will be those that create a high degree of synergy between business and IT. They will break down the barriers between business intelligence, operational systems and office support (collaborative) environments. They will integrate external data directly into their business systems. Business intelligence will become Business Integrated Insight (BI2).
Business Integrated Insight in action
Just as BI set the scope for much of the advances in information usage in the 1990s and the Web the 2000s, big data looks set to dominate information thinking in the 2010s. The term big data covers a multitude of sins. But, here, I focus on one aspect of the topic, sensor-generated data and, specifically, vehicle telematics—the process of transmitting and receiving computer-generated data derived from electronic sensors, typically through an on-board controller / computer and the use of this data to remotely monitor a range of conditions and events occurring in the vehicle.
Electronic environmental sensors are becoming ever cheaper and widespread. These sensors are used to measure specific aspects of the environment or behaviour of a machine or device. Many of us are already familiar with sensors in cell / mobile phones, digital cameras and tablets that detect the device’s location (GPS sensor), orientation, movement and so on. Similar devices have long been fitted to expensive machines, such as aircraft engines, to record many operational characteristics such as fuel consumption, rotation speeds, acceleration, and so on. All of these measurements are passed to an on-board computer and continuously transmitted to the manufacturer’s service centre for ongoing quality monitoring and preventative maintenance. As the technology has dropped in price and cell phone communications have become pervasive, automobile manufacturers have begun to fit such systems, particularly on high-end or fleet models, for similar purposes.
In the mid-2000s, automobile insurers began to trial the use of this data to monitor driving behaviour with a view to “pay-as-you-drive” insurance policies. Full-fledged systems have since been rolled out in various countries. Typical data gathered includes time of use, mileage driven, speed, sudden accelerations and braking, as well as accidents. Based on driving behaviour, premiums are adjusted up or down or discounts offered, often with near-immediate effect—an entirely new business model for automobile insurance. We clearly see that these products can only be offered using technology previously unused in this industry. Also, data that was collected for one technical purpose has been applied to another, innovative and process-changing idea.
Business Integrated Insight—What It Means for 2012 and Beyond
From the point of view of the biz-tech ecosystem and BI2, a number of messages are clear.
First, we’ve seen a consistent move in BI towards operational BI—analysis and action-taking based on near real-time data—over the past decade. While this will continue, it will, at best, only partially meet demands for extreme business innovation. New data sources, big data in most cases, will be used operationally to drive new business processes. Business and IT must cooperate closely to understand how new information and technologies can be used to create new business processes and how such changes can be incorporated into existing IT systems—the interdependence, reintegration and cross-over characteristics described above.
Second, the distinction between what we traditionally call “operational” and “informational” processing is becoming increasingly unclear as the business demands real-time reaction to a rapidly changing environment. This demand, together with exploding data volumes, severely restricts the old approach of copying / cleansing data into a data warehouse and data mart environment. Of course, that approach will remain valid for statutory and regulatory reporting where the highest level of accuracy and consistency is mandatory. But, beyond that, minimising the number of copies of data and accessing one, original data set for multiple purposes will be the norm. This is discussed in a number of papers on BI2 available on my website.
Third, not only are data volumes increasing rapidly but much of this “new” data we will be handling differs in some key characteristics from that which we traditionally used. The new data sources are external and of ill-defined quality. They have very different and often less formalised structures. They have been collected for other purposes than how we want to use them. Ownership and privacy will be serious concerns. I’ve discussed many of these characteristics and their implications in “The Seven Faces of Data” published in November 2011.
So, what specific actions should data management professionals take in 2012? There are many, of course! But, three important innovations top the list:
1. Evaluate emerging data and content stores. Relational databases are no longer the only answer—if they ever were.
2. Investigate virtualisation technologies. Remote, multi-purpose access to information will become the norm.
3. Re-engage with the business. IT must become an equal partner in the vision and implementation of business innovation to maximise the benefit of new technology.
Assuming that the world doesn’t end in December, we’re in for exciting times in business and IT!