Use the “less is more” principle in analytics; it applies as it does to almost anything in life!

Having spent my career as a analytics leader creating solutions, I can tell you that the best analytics capabilities and solutions need not be rocket science. Do not let anyone convince you otherwise because that simply is not true. There is a general tendency these days to ‘over engineer’ analytic solutions. Further, in my observation, there is a direct relationship between the availability of resources and the extent of ‘over engineering’; i.e., the more the resources the more the tendency to ‘over engineer!’

Oxford Dictionary lays out the definition of ‘Analytics’ as “information resulting from the systematic analysis of data or statistics.” From the chart (source: Google Books Ngram Viewer) the mention of the term “analytics” has geometrically exploded in the past decade. It is needless to say that the

advent of ‘big data’ is the accelerator here. Having said that however, the components of ‘analytics’ have fundamentally not changed from the time the term was first mentioned in 1860’s and more so in the post computer era of the 60’s. The components of analytics still remain; descriptive, diagnostic, predictive and prescriptive. This decomposition has not changed whether it be “small data” or “big data”. The only difference is that leveraging technology in the age of ‘big data’ means we can conduct descriptive, diagnostic, predictive and prescriptive analytics much more efficiently than ever in human history.

So, let us get into my earlier contention that we tend to ‘over engineer’ solutions. This situation has particularly exacerbated in the ‘big data’ world. Technology gives us the opportunity to leverage big data like never before. However, we are distracted by the shiny bobble and have stopped making analytics about the ‘first principle(s)’, i.e., ‘describing’, ‘diagnosing’, ‘predicting’ and 'prescribing.' If applied appropriately, we should understand how stimulus effects outcomes. Simple, right? However, analytic solutions built at scale, in my observation, are hardly ever simple! There are ‘terabytes’ of data extracted, transformed and stored for diagnosis that never sees the light of day. Because storage and computing are getting cheaper this problem is multiplying at geometric rates inside organizations as we speak. The greater the resource, the faster the multiplication. This is leading to a plethora of analytic deliverables that are at the same time accurate and virtually ineffective in driving clarity of business decisions. In my discussions with numerous business leaders the term “overwhelmed” is used to describe this phenomenon. They are basically ‘drinking from the hose’ and we, in the analytics fraternity, need to step up, take accountability and fix this.

Here is my theory; business leaders are overwhelmed with analytic solutions that are being ‘over engineered’ because we are taking ‘ALL’ the data, whether we need it or not, describing ‘ALL’ of it, diagnosing ‘ALL’ of it, and trying to predict outcomes that might or might not lead to prescriptive recommendations. We should instead start with ‘prescriptions’ we are looking to get, then lay out the predictive models required, this then should push us to go get only the data we need to describe and diagnose. Try it and you will experience what I have in my practice; excellence in analytics!

Here is an example from our history that might bring this to life. President Kennedy, on September 12, 1962, made a speech at Rice University, where he said, “We choose to go to the moon in this decade and do the other things, not because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills, because that challenge is one that we are willing to accept, one we are unwilling to postpone, and one which we intend to win, and the others, too.” Pay attention to his statement, “we choose to go to the moon in this decade…”, and compare this to the fact that on July 16, 1969, Neil Armstrong laid the first human footprint on the surface of the moon. What President Kennedy did was to lead out with the ‘prescriptive’ action of going to the moon by the end of the decade. This then directed 400,000 engineers to design everything using a minimalist approach taking only what they needed and that is why I believe they succeeded in beating the President’s deadline by 168 days! The minimum viable approach is both powerful and empowering to analysts as well. If actual rocket scientists did not ‘over engineer’ then we have no excuse!

Be the team that aspires to describe, diagnose, predict and prescribe on every analysis and every deliverable using the “less is more” principle; it applies to analytics as it does to almost anything in life!