Big Data becomes friendly in 2013

Many organizations are in the phase of evaluating the Hadoop platform . Certainly Hadoop has been the only option to handle large unstructured data for organizations that run their business handling unstructured data like Google, Yahoo.  For others Hadoop positively provides an opportunity to look at data (Dark Data) which they haven’t considered as part of the Enterprise Data Warehouse.

In the process of defining and executing a proof of concept with Hadoop platform, we generally face two challenges which are:

The need for developers to acquire new skills to handle different programming languages related to Hadoop. It’s not easy for a developer who has worked on a GUI based ETL tool like Informatica to work on Hadoop ETL process.

The means to visualize the results from Hadoop, definitely we need outputs which are more than a search engine output

2012 can be seen as the year which brought in lot more tools and utilities related to Hadoop to make things easier…following are the few key releases from major BI vendors

IBM moved up a level and announced on the availability of few integrations which will increase the adoption of BigInsights platform. Some of them include integration of InfoSphere Data Explorer ( recently acquired product Vivisimo) with BigInsights , availability of Applications Accelerators with the BigInsights platform – Machine Data Analytics Accelerator for analyzing machine data and Social Data Analytics Accelerator for analyzing social media data sources like Twitter, Facebook and integration of Cognos with BIgInsights

The Business Intelligence Chasm

The term Business Intelligence was first coined by IBM researcher Hans Luhn (in the IBM Journal of Research and Development, October 1958) and then used in its modern sense in 1989 by then-Gartner analyst Howard Dresner, who defined BI as an umbrella term to describe concepts and methods to improve business decision making by using fact based support systems.Both these definitions were quite prescient for its time – Mr. Luhn’s concept of ‘action points’ in the organization and Mr. Dresner’s reference to ‘business decision making’ ensured that BI has direct business relevance to go along with its very interesting technology façade.

BI & Analytics, in some sense, represents the holy grail of computer based applications, i.e. the use of technology to solve real world business problems. Clearly, there are 2 distinct aspects to BI –Technology and Business and both have to work synergistically to deliver on the overall promise.

As we step into 2013, my contention is that we as BI practitioners are doing fairly well on the technology front by assimilating many of the new developments (In-memory, Appliances, Columnar storage, Big data processing etc.) into mainstream data management, reporting and analytics, while we lack the skills required to integrate all this in the broader business context. Let me substantiate that statement.

Linking Enterprise Master Data with Social MediaData – Social MDM

“By 2015, 15 percent of organizations will have added social media data about their customers to the customer master data attributes they manage in their MDM systems” – Gartner

In my previous post on Collaborative Data Management we analyzed how data governance, quality and Master Data Management (MDM) can be leveraged to bring about a coherent data conscious environment within an organization. But is data that can provide business insights present within organization boundaries alone? Well with the explosion of Social Media the answer is a big ‘NO’.

As a known fact, traditionally the customer buying pattern was analyzed from data gathered from in-house systems and leveraged for selling opportunities. The information that these systems provided are essentially what the customers wanted to provide, in other words the intelligence was limited to the data collected at the Point of Sales. With Social MDM in place the whole approach changes. Following are the recommended action points as to how organizations need to go about with their Social MDM strategy:

• Determine Attributes: Strategize on potential application of different Social Media and identify new attributes
• Integrate Attributes: Linking MDM within Enterprise and Identity on Social Media followed by Enhancing the ‘Golden Copy of Customer Data
• Build Social Intelligence: Take enterprise to the customer on Social Media

Do you have a vision for your Business Intelligence?

A quick search on the internet on Business Intelligence brings up a myriad of results:

a)  Big Data analytics helps to analyze large volumes of data – Is this BI?

b)  Data Virtualization brings together enterprise data from multiple, disparate sources– Is this BI?

c)   Data discovery tools offer agility and high performance for faster data exploration – is this BI?

None of this is Business Intelligence. The term Business Intelligence (BI) is often confused for a technology. However, the technologies that go by the name of BI are only a means to an end. The end still remains the information, insights and actions delivered through this technology platform. This understanding is critical to the success of Business Intelligence.

An often cited metric about Business Intelligence is that more than 50% of BI projects fail. Even as one questions the veracity of the claim, it is but common knowledge that BI projects drag on for a long period without producing concrete business deliverables. One of the primary reasons that this happens and that BI projects are not considered successful is that success in a BI project is seldom established.

It is not often that organizations make a conscious effort to create a business intelligence vision, which underlines the success criteria for the BI platform. In many organizations, BI initiatives start with one section of the business and gradually evolve to cover other areas. When this happens or when there is an attempt to consolidate several independent local BI initiatives together into an organization-wide BI platform without a clear vision and a direction, the outcome is often less than desirable.

The BI vision is essentially a set of clearly defined business objectives and a bunch of relevant metrics to track the progress towards the objectives and the final outcome. It is important to ensure that there are metrics that reflect both internal (mostly relevant at the divisional level) and external effectiveness (relevant at the organization / market level) and that they are aligned with each other and to the business objectives.

There are several industry-standard methodologies such as the Balanced Scorecard, the Performance Pyramid that are available to deduce strategic metrics or performance indicators in alignment to the overall strategy / business objectives.

Do you want your BI program to be success? Do you have a vision for your Business Intelligence?

Oracle R12 Applications Using LoadRunner

The Challenge

We recently load tested our first Oracle R12 release (All modules for nationwide and international wide of Oracle ERP R12). The company was upgrading to R12 from 11.5.8 largely for performance reasons.

We knew we’d be “cutting new ground” with LoadRunner on R12. This became evident with our first testrecord-and-playback, which failed even after finding and fixing all the missing correlations. We raised a ticket with HP (SR# #4622615067), and with their initial help, step by step we overcame all the nuances of coaxing vugen to record successfully, and then creatively working around its inability to recognize the full set of identifiers for a new java ITEMTREE object.

This Practical Solution describes the challenges we encountered and how we overcame them. See the

Technical Issues and Solutions section for the gory details.

Before You Start

As with 11i, remember that you must expose “Oracle Developer Names*” on the instance before

recording. There’s an HP technote on this, but here are the steps:

• Launch Oracle application
• Login with Administrator privileges
• Select the role “System Administrator”. If you don’t have this role, you don’t have the rights to do this
• Select the option “Profile: System”
• Search for the following profile option: “ICX: Forms Launcher” (Search for a specific user to change only the settings for one user);
• Append this string to the end URL: “?play=&record=names”;
• Save and close.

* Failure to do this will result in generic field names that make scripting much more difficult and harder to

maintain.

Lessons Learned

• HP Support can help
• Oracle Metalink documents provide good info on R12 architecture and configuration
• If VUGen stumbles on a new object, try the QTP Object Spy
• R12 can be installed in two distinct communication modes; make sure it’s in the more efficient “Socket    mode” before you beginning scripting
• Porting scripts is a pretty common requirement – save crucial time at “show time” by parameterizing the instance-specific values up-front

Technical Issues and Solutions

 

Issue1: Playback failure on nca_object_action statements

Following the nca_connect statement there are two statements of this form:

nca_java_action (“SR_DUP_GRID_DUP_GRID_0″,)

On playback, the script invariably fails at the first of these.

Practical Solution:

In each script directory, open the default.cfg file and in the [NCA_GENERAL] section add this line:

NCATimerWaitMode=0

Playback will now successfully process nca_object_action statements.

 

 

 

 

Issue2: Java Runtime version and memory setting

Oracle R12 now loads the JInitiator file (runtime program for executing the Oracle java applet) from the

Java JRE, instead of downloading it during the Oracle initialization process or installing it on your PC separately. Also, Forms 10g needs more desktop memory than Forms 6 to perform reasonably.

Practical Solution:

Upgrade the Java JRE to version 1.0.0_05 or later. Moreover, allocate at least 512 MB of memory, using the Java Control Panel, and add the parameter:

–mx512m –Dcache.size=500000000.

Issue3: The correlation of ncx_ticket is different from 11i

In 11i, the correlation rule to correlate the key Oracle Forms session id, ncx_ticket, yields a web_reg_save_param statement with these left and right boundaries:

web_reg_save_param (“ICX_Ticket”, “LB=icx_ticket=”, “RB=’”, LAST);

In R12, a new suffix appears on the LB and a there is a new RB:

web_reg_save_param (“ICX_Ticket”, “LB=icx_ticket&gv15=”, “RB=&“,LAST);

Practical Solution:

In the Recording Settings, modify the OracleApps ‘icx’ correlation rule with the above LB and RB

to enable vugen to properly correlate icx_ticket automatically.

 

 

Issue4: Failure to recognize certain new java objects on record

QTP has issues recording an R12 object called an ITEMTREE which was a list of expandable items that

look like this:

+40907824 ITEM1

+40907839 ITEM2

The following statements resulted after selecting an item during recording, but these would not play back:

nca_tree_select_item (“ITEMTREE_ITEMTREE_0″, “40907824 ITEM1 “);

nca_tree_activate_item(“ITEMTREE_ITEMTREE_0″, “40907824 ITEM1 “);

If you encounter this object, or any other new object that vugen has trouble with, this tip may get you

around the recording limitation.

Practical Solution:

We used QuickTest Pro’s ObjectSpy to examine the ITEMTREE object and found that each menu

item had a reference number that is selectable and plays back correctly. We then modified the

VUgen code with the correct reference numbers:

nca_tree_select_item(“ITEMTREE_ITEMTREE_0″, “409“);

nca_tree_activate_item(“ITEMTREE_ITEMTREE_0″, “409“);

This reference number is listed in the detail log files if you look carefully but they were not obvious

during recording, but we recommend using ObjectSpy if you encounter any other objects that

VUgen doesn’t handle correctly.

 

Issue5: Porting scripts between environments

A common situation is that you must develop scripts in one environment but conduct testing in another.

This requires that your scripts be portable across environments.

Practical Solution:

There are several values that need to be parameterized in order to make your scripts portable.

Parameterize and test these as early as possible to avoid frantic porting at test execution time!

The values to parameterize are:

1. Base URL or app server ip

2. Web server port number which you initially connect to for login authentication

3. Forms server port number, which the nca_connect sues to connect to the Forms server

4. Configuration (“config” value in nca_connect)

5. Module (second ‘path’ value in nca_connect)

Examples of where these are used:

/*initial Oracle launch*/

web_browser (“UnysisERPR12.com:8050“,

DESCRIPTION,

ACTION,

“Navigate=http ://{ app_srv} :{ port_web}/”,

LAST);

/*Forms server connect*/

nca_connect_server (“{app_srv}“, “{port_forms}“,

“module=/ebstop/{module}/apps/apps_st/appl/fnd/12.0.0/forms/US/FNDSCSGN fndnam=APPS

record=names config=’{config}‘ icx_ticket=’.{ICX_Ticket}..’

resp=’AR/CAN_CUSTOMER_MASTER_ADMIN’ …”);