on target
How much analytic are decision makers?
James Taylor recently made some considerations about a post by Stephen Few on visualization tools. Stephen Few is a passionate and skilled expert in quantitative data visualization. His approach, rational and effective presumes that graphics shall help decision makers to understand data, and get from it useful information. I think that everybody agrees with this.
Taylor, on the other side points that most companies “start with the data and go forward from it”. I admit that this is in fact the reality: far from the ideal but the usual case! Most BI projects are an IT initiative in an attempt to aggregate value to the large amount of data collected along time. This explains why most BI tools are highly cosmetic report writers full of features like gauges and pie charts: impressive to nerds but poor choices for decision support.
To make an analogy can you figure an architect that instead of planning a house, based on client needs, says “well, here are some spare bricks, tiles and concrete, let's see what we can make out of them”. Not the ideal architectural process, but widely employed in Latin America to build the so called “Favela”.
I am absolutely not criticizing starting from data and I fully agree with Taylor that this is the common (when not the only) alternative for one simple reason: today's management approach is far away from scientific and even disciplines that should employ a more rational approach are analytically too superficial.
Need a proof? Look at management titles in any library or bookstore: count how many of them are driven by emotional theories, optimism and self-help approach? Lot of blah, blah, blah filled with magnetic words like “win”, “strategy”, “differentiation” but absolutely no quantitative analysis, no simple and practical formula or practical step-by-step rational process.
Where has scientific management gone? Where has operations research gone?
But let's stay in a subject that is supposed to be highly quantitative and is (or better “has been”) an hot topic: performance management, and let's consider a common KPI: sales force responsiveness.
In a well known KPI book I read: Average time from custom enquiry to sales team response. If you go on reading an extensive list of KPIs you find lot similar measures. Nothing against this book: I am sure you can count dozen of alike cases in most KPI dashboards or scoreboards. Averages, ratios, ratios, averages ...
Can you spot any problem with this measure? No?
Supposing that an acceptable value would be 24 hours (more than one day would frustrate most buyer patience), consider the following set of cases:
5, 41, 29, 32, 28, 4, 31, 29, 28, 27, 28, 1, 34, 67, 29, 2, 30, 2, 8, 40, 2, 21, 32, 36, 27, 1, 36, 1, 5 hours.
Average is 22.62 hours, still in the “green” range of the dashboard if maximum is 24 but... about 65% of your prospects needed more than one day to get the response and will possibly choose another supplier!
Using the wrong measures is the best way to kill any performance management application. As soon as management perceives that KPIs do not reflect actual performance the dashboard enters an abandon phase (it is not dismissed only because some manager spent her budget to build it).
So let's find a better measure. Taking the maximum, instead of average, would be still less expressive (a single high value would compromise any good value). A simple is to measure the proportion of cases not exceeding a predefined value (24 hours for example). For example 85% of cases shall be responded in less than 24 hours.
There is a more sophisticated and precise solution using fuzzy math (please post a comment if you are interested) but that's not the goal of this post. My intention was simply to demonstrate that analytics is something more than simply aggregations averages and ratios.
In next post I will show an even more curious frequent and case from Web Analytics.
Interested in partnership?
Would like to try or coach this technique with your clients, do you need any additional or technical detail? Please let me know! Component Bases Solutions has great interest in partnership with consultants. We can help you automate your proposed solutions in a very short time. We can also help to increase your visibility through links from many management tools we make freely available on the Web. Please contact for more detail.
Did you like this post?
Please, spend a few seconds and click on button below to socially bookmark it or tweet it. It will help to promote this blog and improve its content, Thank you!
|
 |
 |
Posted at 10:42PM Mar 23, 2010 by admin in General | Comments[4]
Posted by BizSugar.com on March 24, 2010 at 10:35 AM UTC #
If that was the data I was presented with, I would follow up with a series of questions, what does this look like over time?, what trends, or influencing factors can we see with additional fields, like sales person?, what does this hours value really mean, is it working hours or simple hour diff?, why is there a wide range of values? how was this collected? and many other questions, and questions based on the answers.
I believe your solution uses sound business logic, but leaves out the understanding or the data itself.
I agree that simply starting with data and going forward is not enough, but but just applying your business logic without fully understanding your data could also lead to inaccurate results, or lack insight to make decisions.
Taking your example data set, what if that 67 hour data point crossed a weekend, if you changed to looking at working hours, that would become 3 hours. Also, what action do you take after knowing that 61% of the data points are meeting your expected business requirements? The 61% is meaningful, but not actionable. I agree with what you said, but it feels like an incomplete analysis.
I am also interested the fuzzy math you are referring to.
Posted by Joe Mako on March 25, 2010 at 01:24 AM UTC #
Thanks for the comment and the opportunity to clarify. Being an effectiveness measure of client perspective hours are probably hours as “seen by client”, excluding not applicable time windows like week-end (from client point of view). I say “probably” because how time is computed depends on specific policies and is not the focus of the post.
This is in fact an example to demonstrate how superficial measures (like a simple average) can produce poor results. No matter how hours are computed (we expect they are correctly computed and understood), taking their average is a poor choice.
From business point of view knowing that my average response time is 22 hours tells me nothing, but knowing that 39% of my clients were probably frustrated by the agility of my sales force makes, at least to me, much more sense. It is also actionable, from a management point of view: either improve or somebody will be fired!
That does not mean that you do not need more detailed data before taking any action, and for this you will rely on analytic capabilities of your application (drill down/through) but again, that extrapolates the goal of the post that was only to show the difference between overused KPI averages and a better measure.
I'll address fuzzy math in another comment.
Posted by Franco Graziosi on March 25, 2010 at 09:49 AM UTC #
Fuzzy Math
What is “reasonable” as expected response time for a client or prospect may be not reasonable for another one. It is like saying that a person is young or old, it depends from the point of view: to a teenager an 40 years old man can seem old. The same person can seem young to an aged person.
But almost everybody agrees that somebody under 20 years is young ans over 40 is not young any more!
Shortly fuzzy math understand that logic is not so crisp and instead of employing false/true boolean values we use values in 0-1 range. Thus, instead of defining “young” as “age under 30” we define it as 1 if under 20, 0 if over 40 and something between 0 1n 1 in between. The in-between is usually linearly interpolated (a simplification) but other alternatives are possible.
Back to our case (again supposing that the number of hours correctly reflect the perceived elapsed time from client point of view) after analyzing polling client expected response time, you can determine a function that maps response time to a 0-1 service-level measure and in this case, yes, take the average of those values.
Posted by Franco Graziosi on March 25, 2010 at 10:12 AM UTC #