A Proactive Approach to Building an Effective Data Warehouse

“We can’t solve problems by using the same kind of thinking we used when we created them.” – The famous quote attributed to Albert Einstein applies as much to Business Intelligence & Analytics as it does to other things. Many organizations that turn to BI&A for help on strategic business concerns such as increasing customer churn, drop in quality levels, missed revenue opportunities face disappointment. One of the important reasons for this is that the data that can provide such insights is just not there. For example, to understand the poor sales performance in a particular region during a year, it will not just help to have data about our sales plan, activities, opportunities, conversions and sales achieved / missed, it will also require understanding of other disruptive forces such as competitors promotions, change in customer preferences, new entrants or alternatives.

Thomas Davenport, a household name in the BI&A community, in his book ‘Analytics at Work’, explains the analytical DELTA (Data, Enterprise, Leadership, Targets and Analysts), a framework that organizations could adopt to implement analytics effectively for better business decisions and results. He emphasizes that besides the necessity of having clean, integrated and enterprise-wide data in a warehouse, it is also important that the data enables to measure something new and important.

Big Data becomes friendly in 2013

Many organizations are in the phase of evaluating the Hadoop platform . Certainly Hadoop has been the only option to handle large unstructured data for organizations that run their business handling unstructured data like Google, Yahoo.  For others Hadoop positively provides an opportunity to look at data (Dark Data) which they haven’t considered as part of the Enterprise Data Warehouse.

In the process of defining and executing a proof of concept with Hadoop platform, we generally face two challenges which are:

The need for developers to acquire new skills to handle different programming languages related to Hadoop. It’s not easy for a developer who has worked on a GUI based ETL tool like Informatica to work on Hadoop ETL process.

The means to visualize the results from Hadoop, definitely we need outputs which are more than a search engine output

2012 can be seen as the year which brought in lot more tools and utilities related to Hadoop to make things easier…following are the few key releases from major BI vendors

IBM moved up a level and announced on the availability of few integrations which will increase the adoption of BigInsights platform. Some of them include integration of InfoSphere Data Explorer ( recently acquired product Vivisimo) with BigInsights , availability of Applications Accelerators with the BigInsights platform – Machine Data Analytics Accelerator for analyzing machine data and Social Data Analytics Accelerator for analyzing social media data sources like Twitter, Facebook and integration of Cognos with BIgInsights

The Business Intelligence Chasm

The term Business Intelligence was first coined by IBM researcher Hans Luhn (in the IBM Journal of Research and Development, October 1958) and then used in its modern sense in 1989 by then-Gartner analyst Howard Dresner, who defined BI as an umbrella term to describe concepts and methods to improve business decision making by using fact based support systems.Both these definitions were quite prescient for its time – Mr. Luhn’s concept of ‘action points’ in the organization and Mr. Dresner’s reference to ‘business decision making’ ensured that BI has direct business relevance to go along with its very interesting technology façade.

BI & Analytics, in some sense, represents the holy grail of computer based applications, i.e. the use of technology to solve real world business problems. Clearly, there are 2 distinct aspects to BI –Technology and Business and both have to work synergistically to deliver on the overall promise.

As we step into 2013, my contention is that we as BI practitioners are doing fairly well on the technology front by assimilating many of the new developments (In-memory, Appliances, Columnar storage, Big data processing etc.) into mainstream data management, reporting and analytics, while we lack the skills required to integrate all this in the broader business context. Let me substantiate that statement.

Linking Enterprise Master Data with Social MediaData – Social MDM

“By 2015, 15 percent of organizations will have added social media data about their customers to the customer master data attributes they manage in their MDM systems” – Gartner

In my previous post on Collaborative Data Management we analyzed how data governance, quality and Master Data Management (MDM) can be leveraged to bring about a coherent data conscious environment within an organization. But is data that can provide business insights present within organization boundaries alone? Well with the explosion of Social Media the answer is a big ‘NO’.

As a known fact, traditionally the customer buying pattern was analyzed from data gathered from in-house systems and leveraged for selling opportunities. The information that these systems provided are essentially what the customers wanted to provide, in other words the intelligence was limited to the data collected at the Point of Sales. With Social MDM in place the whole approach changes. Following are the recommended action points as to how organizations need to go about with their Social MDM strategy:

• Determine Attributes: Strategize on potential application of different Social Media and identify new attributes
• Integrate Attributes: Linking MDM within Enterprise and Identity on Social Media followed by Enhancing the ‘Golden Copy of Customer Data
• Build Social Intelligence: Take enterprise to the customer on Social Media

Do you have a vision for your Business Intelligence?

A quick search on the internet on Business Intelligence brings up a myriad of results:

a)  Big Data analytics helps to analyze large volumes of data – Is this BI?

b)  Data Virtualization brings together enterprise data from multiple, disparate sources– Is this BI?

c)   Data discovery tools offer agility and high performance for faster data exploration – is this BI?

None of this is Business Intelligence. The term Business Intelligence (BI) is often confused for a technology. However, the technologies that go by the name of BI are only a means to an end. The end still remains the information, insights and actions delivered through this technology platform. This understanding is critical to the success of Business Intelligence.

An often cited metric about Business Intelligence is that more than 50% of BI projects fail. Even as one questions the veracity of the claim, it is but common knowledge that BI projects drag on for a long period without producing concrete business deliverables. One of the primary reasons that this happens and that BI projects are not considered successful is that success in a BI project is seldom established.

It is not often that organizations make a conscious effort to create a business intelligence vision, which underlines the success criteria for the BI platform. In many organizations, BI initiatives start with one section of the business and gradually evolve to cover other areas. When this happens or when there is an attempt to consolidate several independent local BI initiatives together into an organization-wide BI platform without a clear vision and a direction, the outcome is often less than desirable.

The BI vision is essentially a set of clearly defined business objectives and a bunch of relevant metrics to track the progress towards the objectives and the final outcome. It is important to ensure that there are metrics that reflect both internal (mostly relevant at the divisional level) and external effectiveness (relevant at the organization / market level) and that they are aligned with each other and to the business objectives.

There are several industry-standard methodologies such as the Balanced Scorecard, the Performance Pyramid that are available to deduce strategic metrics or performance indicators in alignment to the overall strategy / business objectives.

Do you want your BI program to be success? Do you have a vision for your Business Intelligence?