Friday, December 14, 2007

Business week piece on R&D

Untill I get off the work Pressure, an interesting piece from BusinessWeek.


Why are companies slow to make full use of their offshore brainpower? Because managing a globally integrated product-development team is difficult. One obstacle is "not invented here" syndrome: Developers at headquarters tend to look down on suggestions from offshore affiliates. Internal competition between different R&D teams within the same company remains rife.On top of this, many companies lack harmonized work processes and systems,and few have enough staff with international experience. The bottom line: "Few companies are currently reaping the gains from this globalization," ---Booz Allen-INSEAD stated.

Thursday, November 15, 2007

The Generic BI Architecture

The chart below shows more or less an architecture of Busines Intelligence delivery right from the data generations Stage to the Report Generation & decision Support. The flowchart is divided into 3 phases, Phase1,Phase2,Phase3. I shall discuss Each of them very briefly (Click on the pictures for better viewing of the text in the diagram)



• The phase 1 of the diagram involves collecting the Transaction data from various operational data sources (ODS) that are deemed essential for records storage by the business.

• The data from all these various sources is extracted to one region for further manipulation. This is a very complex region where the data is brought on platform from various sources.

• Here the data may be extracted real time, hourly basis, daily, weekly monthly etc depending on the business need & other constraints.

• The phase 2 starts when all the data that is in the single platform is then made into a single format & made relevant in terms of removing all the inconsistency that may be prevailing.

• Again even this complex process is extremely important & is termed as data integration.

• Once the data is made consistent it is further normalized as per certain rules & standards of data warehousing & as required by the business.

• The data is stored constantly in the Data warehouse & is segregated in to different tables for relevance while storing.

• There are so many other complex processes involved here & in the previous phases, I wouldn’t be going deep into it.

• Once the data storage part is complete then comes the reporting part.

• Cubes are created for faster retrieval from the data warehouse & to view its multidimensionality.

• Data marts are created so that the relevant stakeholders can extract the data or do reporting from the data relevant only to their functional unit.

• Scorecards, Dashboards, & other reporting can be done directly from data warehouse or from data marts or from cubes as may be the need.

• The reporting part also involves a certain amount of analytics but most of this is very limited. Unless the end user is very well versed with the OLAP tool, data, analytics a very good outcome for supporting ad-hoc & complex business decisions become very difficult & most of the BI projects are not able to deliver this. This not only due to the tool but also due to several other factors. Thus versatility needed for the analytics team gets hampered & thereby the BI project outcome. All the million dollars go in a ditch.

Monday, November 12, 2007

The bewildered state of a Business Analyst!!!!!!!!!!!!!!!!!

Time again I have tried to emphasize what a business analyst is & should be doing. In my previous topic “Critical Skills a Business Analyst must have” I have given a framework on what is expected of BA based on the Gartner theory & industry expectations. It’s sad to see most of the generation getting swayed by terms & jargons & trying finding what they want in the wrong place & at last ending up smelling socks. Well the industry in the mother of all the confusions!! For those of you who still feel that Business Analyst is obscure, I think this should help.

• Systems analysts who generally analyze the requirements & thereby find the system needs to fulfill such requirements. Act as business/system liaison between business and application developers.

•Requirements analysts - who generally do the requirements elicitation & understand what people say and to detect what they might be hesitant to say blah blah blah

• Business system analysts these are generally the victims formed by those companies who believe in MPD (Multiple personality disorder) & cost cutting. They want you to do both the work of requirement & system analyst & a little of other things also until you go completely mad & run out of the company heels over head. Oh well did you think the fancy title is good? Try your luck.

• I love this role for all its challenges involved in practicing & living the unbridled forces that influence business. Simply put Business analysts are responsible for understanding the business and making recommendations for sustainability, risk mitigations, Business improvement & profits blah blah blah

For those of you who need a strong dosage of analytics I’d recommend be sure you understand what continuous & discrete data is, before you click on www.businessanalytics.in
For those of you who think have strong appetite for Greek & Latin with all the animation in between, if you would like to work in a warehouse & have the nerve to load & unload bulks of data & for more gyaan on this log on to http://www.keysoft.co.in/

Disclaimer - Both the institutes train in different domains, please exercise diligence before enrolling yourself into these & first decide which one you fit in.

Friday, November 9, 2007

Butterfly Says Happy Diwali To you !!!!!!


”One day, a small opening appeared on a cocoon; a man sat and watched for the butterfly for several hours as it struggled to force its body through that little hole. Then, it seems to stop making any progress.It appeared as if it had gotten as far as it could and it could not go any further.So the man decided to help the butterfly: he took a pair of scissors and opened the cocoon. The butterfly then emerged easily.But it had a withered body, it was tiny and shrivelled wings.The man continued to watch because he expected that, at any moment, the wings would open, enlarge and expand, to be able to support the butterfly’s body, and become firm. Neither happened!In fact, the butterfly spent the rest of its life crawling around with a withered body and shrivelled wings. It never was able to fly. What the man, in his kindness and his goodwill did not understand was that the restricting cocoon and the struggle required for the butterfly to get through the tiny opening, were God’s way of forcing fluid from the body of the butterfly into its wings, so that it would be ready for flight once it achieved its freedom from the cocoon. Sometimes, struggles are exactly what we need in our life.If God allowed us to go through our life without any obstacles, it would cripple us. We would not be as strong as we could have been. Never been able to fly.

I asked for Strength...
And God gave me difficulties to make me strong.

I asked for Wisdom...
And God gave me Problems to solve.

I asked for prosperity...
And God gave me a Brain and Brawn to work.
I asked for Courage…..
And God gave me obstacles to overcome.

I asked for Love...
And God gave me Troubled people to help.
I asked for Favors...
And God gave me Opportunities.

“I received nothing I wanted...
But I received everything I needed."
Live life without fear, confront all obstacles and evince that you can overcome them.
“I hope this little message on this Diwali Season would bring in the understanding that why we come across difficult situations in our life before we reach our ultimate destination............”
I wish u a very HAPPY DIWALI to you and to your family members !!
May god bless this country, with peace & prosperity – Data Smith


(Courtesy – UIS HR)

Wednesday, October 31, 2007

Critical skills that a BA / BI team Must Have

As I have reiterated that all businesses are and should be dynamic to survive the storm of this competition, Business initiatives are invariably affected by the environment in which they operate, and they are shaped by many external forces including government policy, international relations, geopolitical stability, economics, market trends, terrorism, consumers, social changes, media etc. They are also influenced by several internal forces that influence the context, in which these initiatives operate, processes, staff turnover, corporate restructuring, policies, people, work atmosphere etc.

While the CEO / Decision making Body Decides on a strategy, an BA should have some skills in anticipating potential outcomes of such initiative / strategy / change, also see it in the light of the current market & company placements (Internal & external forces) & take necessary steps to protect the interest of the stakeholders.

Anticipating changes, analyzing potential impact, avoiding derailment of the initiative form one area where the BA / BI team / BI component need to be extremely strong, consistent & coherent.

BI teams, BA’s are key people in realizing a company’s vision & providing solution to the challenges that a company or an initiative has, will have & is having. More importantly the core challenge is in accurately answering the business questions that arise at different levels, articulate complex statistical & data jargons to a language understandable by the business, BU managers. & thereby induce radical changes.

Offering critical insights to both internal & external forces that affect initiatives, planning for operational excellence, having strong ability to analyze & see details, but at the same time conceptualize the big picture.* (This is a very important & interesting topic which I shall discuss in detail later on) In all aspects of business they should increase the efficiency, accuracy, speed of decision making in all the conventional, practical & creative decision making areas.

Finding new data patterns of the domain applying logic, new ideas, intuition (which comes by domain experience) should fuel the company’s prospects into a BLUE OCEAN & find potential un-dominated areas to excel, is also important for a BI team.

We should be the people / team with technical, Business, analytical, & strong communications skills (mails, verbal, written, non written, presentations etc).
This should help the team / BA / BI to frame business problems in understandable terms to help manage business implications, voice concerns on a strategy/initiative which may be at stake if the BI is not heard.

Thursday, October 18, 2007

I don’t want to be a BA!!!!!!!!!!!-----3

By the time the data was received it was more than 2 hours.
Further analysis & presentation of the data was delayed.
The decision taking time was increased & a client meeting was impacted.
Loss to the business was intangible.


This is a very small example of Cross functional BI delivery system failure or lack of coordination between the components of the BI delivery system.

On a larger scale I had another friend of mine who told me a similar experience in the marketing domain were the BI team had forecasted & confirmed some turbulence in a given area but since the same wasn’t cascaded enough to the field executives a huge amount of financial loss was suffered. These are just some of the examples that I have seen but I’m sure there a million of such instances in almost all the sectors were the BI team is not well established or matured. Even in the case of the BI team being matured the corporate system wouldn’t simply integrate with the input & output part of the BI teams. This is the root cause for the failure of most of the BA projects around the world. However there are other reasons like lack of skill sets, wrong tests carried out for decision making, wrong data considered, no correct data available various other reasons.


One of my friend said “We need to have good modelers, analysts, marketers, business managers etc. And I think, that is what any Analytics Team should be made of…”
I would tweak this to say A BI teams Critical Components (In terms of skill sets) are

Data Collection, warehousing & storage Analysts.
Data storage, manipulation, archiving Analysts.
Data mart, ETL, Reporting etc Analysts.
Query Optimization, emergency response Decision support analyst.
Statistical analysis specialists, Business analyst


I’m sure you will strongly agree that every business is highly & constantly changing itself & adapting to the highly competitive global market. Not only this but every company need to sustain its growth, withstand market forces, withstand & survive economic, financial & various other storms. Also given that most of the sectors are getting saturated, it’s very important for every company to apply new dimensions, drive innovation & plunge deep into Blue waters.


All the above compelling factors drive needs/questions at various levels of a company that are critical to its success


Thus the BI needs to help the business make tactical decisions, modify, tune or realign business strategies & processes to gain competitive advantage or in countering any inevitable market forces that could tamper the business goals. Most importantly thousands of questions arising at various levels of business are to be answered most consistently, accurately, coherent to the business language and of course “I need the report before you take your next breathe”
The process of supporting decision making is iterative in nature & is generally catered by the ad-hoc reports, scorecards, etc which needs hands on approach, thanks to BI tools which have become more friendlier. May it be SAS, Cognos, Business objects etc all of them try to cater in user friendly authoring of reports, execute queries, and perform analysis and various other visualizations. So it’s a fairly easy job to handle huge volumes of data now than ever before. But this is just not what BI is expected of.

Any company that knows its customers & business is already well aware of many important high pay off patterns that its employees have observed over the years.
What a BA & data mining should also and can do is confirm such empirical observations & find new subtle & dynamically changing patterns that could yield steady & incremental improvements in line with the business strategy & goals (Plus the occasional breakthrough / insight)

The critical cross functional flow pattern every BI team’s should have according to me is something like the following.

(Before I talk about the long chart here is a Kit Kat break)

Plight of Business & BI team

I need this, I need that, and it should be this but not that….
It should only focus on this with positive impact……
Tell me who is first; tell me who is best….
I need this, I need that…………
I need it before I die…………..

What will happen there….. If I do this here,
Where will the impact be? If that was done
What will happen if it’s not done?
I need this, I need that…………
I need it before you take the next breathe…

Sir I don’t understand, Madam you aren’t listening,
We can’t get this done, server is not happening,
This is down & that is down,
He is on off & she is laid off ;)
I need this, I need that…………

OOPs & loops I went bananas….
What the fuck I’m stuck……..
This formula wouldn’t work with this data!!!!!!!!!!
Oh no it’s this ways... ok I gotcha!!!!!!!!!
We got it sir, we’ve done it….


Oh your great….this is great…..
But can you simply tell what our fate……is…
I don’t know what regression…..is
I’m only concerned with what our profit progression …….. Is
I need this, I need that…………
I need it before we die…………..

A typical conversation when a BU manager asks a report when sometimes we can make no sense of what they say….. (Yogesh this is for you, remember Jonathan Coplin Beau).
Sometimes we get stuck in puzzles that seem to be the greatest mystery for even Microsoft’s development team, at the end we realize it was just a simple thing all was needed was different eye. We send high info reports with all the statistical or other jargons when the BU manger goes bonkers while he doesn’t make any sense out of it…even if he does he still has questions….. So on & so forth………..

Wednesday, October 17, 2007

I dont want to be a BA!!!!!!!!---- 2

Well no I don’t think so; I have seen a person doing just the ETL, though not a part of the Business Intelligence or Business analyst team is called a BA.
People doing requirements drafting for a development team are BA’s so on & so forth there at least a 100 interpretations of the TAG “Business Analyst” at each of them are unique to each company & specialize in their skill sets.

What is the repercussion’s of this???

Hiring mistakes done by the companies who do not sufficiently mine the prospective skill sets.

Inconsistency between the BA team & the management in delivering what is needed by the business. This is a disaster!!!!! This would badly affect the purpose of having a BA team thereby affecting the companies, decision making/supporting systems & processes.
Here is a simple Case parallel to the analogy of Business analysts.


A report was to be produced which needed to support the training needs for Service desk analysts during the off peak/non business hours. It was not just a report but also confirmation on some of the issues that are reported during the off-peak hours & the analyst needed to also confirm the pattern, impact & other correlations. This was needed within 2 hours of the request being made & was on high priority.
So there was a definite amount of analytics that had to go into the report before any decision could be made.

There were 2 separate teams; one was that of data Analysts & another of operations analysts. The DA’s are supposed to be good at SQL queries, extraction & so on. The operations analyst team was supposed to be good in analytics. But there was no coordination between these two teams & were functioning separately. Unless the data arrived from the DA’s the Ops analysts could not go ahead.
However this roadblock was broken when one of the ops analysts sought DA’s assistance in a politically correct manner. Surprisingly the entire DA team was on in solving the puzzle as the query had to be developed. It was fairly easy question but drawing a accurate query within a short time was the challenge.

The data that was needed had to be after 7pm up to 6 am on Monday to Fridays. + Starting 7 PM Friday continuously to 6AM Monday morning.

There were 2 queries that were developed one was by the DA team & one was by a person who hailed from the DA team (Thanks Vidya) but at that time worked with the ops analyst team. Below is the query developed by the DA team.

SELECT * , --teadd(n (24*60-DATEPART(HOUR,EST_OPEN_TIME))+(7*60),EST_OPEN_TIME),24-(DATEPART(HOUR,EST_OPEN_TIME)) AS HR,60-(DATEPART(MI,EST_OPEN_TIME)) AS MI,60-(DATEPART(SS,EST_OPEN_TIME)) AS SS, CONVERT(VARCHAR(4),24-(DATEPART(HOUR,EST_OPEN_TIME))) +':'+ CONVERT(VARCHAR(4),60-(DATEPART(MI,EST_OPEN_TIME)))+':'+CONVERT(VARCHAR(4),60-(DATEPART(SS,EST_OPEN_TIME))) as done, CONVERT(VARCHAR(4),7+(24-(DATEPART(HOUR,EST_OPEN_TIME)))) +':'+ CONVERT(VARCHAR(4),60-(DATEPART(MI,EST_OPEN_TIME)))+':'+CONVERT(VARCHAR(4),60-(DATEPART(SS,EST_OPEN_TIME))) as did FROM VW_INCIDENTS_HISTORIC_BLRWHERE COMPANY = 'xxxxxxx'AND EST_OPEN_TIME>='20-FEB-07' AND EST_OPEN_TIME<'20-MAY-07'--AND DATEPART(HOUR,EST_OPEN_TIME)>=18 --AND datepart(HOUR,(dateadd(n, (24*60-DATEPART(HOUR,EST_OPEN_TIME))+(7*60),EST_OPEN_TIME)))<=07 --AND (CONVERT(VARCHAR(4),7+(24-(DATEPART(HOUR,EST_OPEN_TIME)))) +':'+ CONVERT(VARCHAR(4),60-(DATEPART(MI,EST_OPEN_TIME)))+':'--+CONVERT(VARCHAR(4),60-(DATEPART(SS,EST_OPEN_TIME)))) AND DATEPART(HOUR,EST_OPEN_TIME) NOT BETWEEN 7 AND 18AND DATEPART(DW, EST_OPEN_TIME) IN (2,3,4,5,6)UNIONSELECT *, --dateadd(n (24*60-DATEPART(HOUR,EST_OPEN_TIME))+(7*60),EST_OPEN_TIME),24-(DATEPART(HOUR,EST_OPEN_TIME)) AS HR,60-(DATEPART(MI,EST_OPEN_TIME)) AS MI,60-(DATEPART(SS,EST_OPEN_TIME)) AS SS, CONVERT(VARCHAR(4),24-(DATEPART(HOUR,EST_OPEN_TIME))) +':'+ CONVERT(VARCHAR(4),60-(DATEPART(MI,EST_OPEN_TIME)))+':'+CONVERT(VARCHAR(4),60-(DATEPART(SS,EST_OPEN_TIME))) as done, CONVERT(VARCHAR(4),7+(24-(DATEPART(HOUR,EST_OPEN_TIME)))) +':'+ CONVERT(VARCHAR(4),60-(DATEPART(MI,EST_OPEN_TIME)))+':'+CONVERT(VARCHAR(4),60-(DATEPART(SS,EST_OPEN_TIME))) as did FROM VW_INCIDENTS_HISTORIC_BLRWHERE COMPANY = 'xxxxxx'AND EST_OPEN_TIME>='20-FEB-07' AND EST_OPEN_TIME<'20-MAY-07'--AND DATEPART(HOUR,EST_OPEN_TIME)>=18 --AND datepart(HOUR,(dateadd(n, (24*60-DATEPART(HOUR,EST_OPEN_TIME))+(7*60),EST_OPEN_TIME)))<=07 --AND (CONVERT(VARCHAR(4),7+(24-(DATEPART(HOUR,EST_OPEN_TIME)))) +':'+ CONVERT(VARCHAR(4),60-(DATEPART(MI,EST_OPEN_TIME)))+':'--+CONVERT(VARCHAR(4),60-(DATEPART(SS,EST_OPEN_TIME))))AND DATEPART(DW, EST_OPEN_TIME) IN (7,1)


And below is the query developed by the person from the DA team but was now in the Ops Analyst team.



select numberprgn , month(open_time), datepart(dw, open_time), datepart(hh,open_time),
case
when (datepart(hh,open_time) >=7 and datepart(hh,open_time) <=18 and datepart(dw, open_time) between 2 and 6) then 'Buss'
else 'Non'

end as 'Classi'
from VW_INCIDENTS_HISTORIC_BLR
where company like 'xxxxxxxxxxx'
and open_time>='1-Jan-2007' and open_time<='1-Sep-2007'


After hrs query –

Time Considred

Mon – Fri 7am to 6 PM – Buss Hrs
Mon - Fri - 6 PM – 7 am - non Buss
Sat - Sun full day – non Buss

The former took a lot of time to be written, was lengthy to comprehend, execute etc. unlike the latter.

By this time the 2 hour deadline of delivering the report is already breached, presumably there is a negative business impact.

What do we infer from this event?

( would follow next……….)

Monday, October 15, 2007

I dont want to be a B.A!!!!!!!!!!----1

I write this rising from the frustrations and the plight after finishing a BI (Business Intelligence) course from the only institute in Bangalore, and after doing enough reading about the much hyped BI & its expected outcome, its failures in the market. Who is a Business analyst? What is a Business intelligence team? Everyone seems to have their own customized shasthra about the BI & BA (Business analyst).


some understanding about BI per Gartner………..

A core topic of research coined by Gartner in late 1980’s, BI is a USER CENTERED / DEPENDENT PROCESS who would access, explore & analyze organizational data. This should & must lead to developing insights & better understanding of the business genetics, forces that influence, govern the growth etc. This in turn would lead to improved & informed decision making in a broader sense. In today’s highly competitive scenario, the BI doesn’t stop still decision supporting but it goes beyond that in predictions, forecasting, modeling & so on.

I see that the BI’s expectations are really high & they seem to be in line with the thinking process of chief in command of a project/unit/company.
But is this true in the real market scenario ?
Are business anlaysts delivering per what i have mentioned above?



(More would follow)