Working on Big Data in business? Hire a physicist
In a post I wrote on big data in November, big data was outlined and explored based on an event delivered by Xconomy in Boston.
A theme that I recognized during that event was that big data challenges require resources that are scarce in the marketplace. From the ability to write the code required to deal with these huge data sets to data scientists charged with making sense of the data and helping derive value from it, specialized professionals will need to work alongside business analysis, program and project managers to deliver value within standard business operating cycles (annual budgets, processes, etc.).
The McKinsey Global Institute, in a comprehensive paper from June 2011 entitled, Big data: The next frontier for innovation, competition, and productivity, put forth that a significant shortage of talent will constrain the derivation of value from big data.
A recent Wall Street Journal blog post, Data Scientists Will Unlock Big Data’s Promise, also highlighted the specialized skill sets needed by data scientists – “Data scientists should have a hybrid set of skills: the IT skills that are necessary to deal with and analyze vast amounts of data; and the subject matter skills needed to know which valuable business insights can be extracted from the data, and how to best frame the questions and build the right model that will reveal these insights.”
These important skills highlight scientific methods learned in school including systematic observation, gathering information and empirical testing or measurement.
The Wall Street Journal article points to an article by Mike Loukides of O’Reilly Media who suggests that “the best data scientists tend to be physicists and other scientists.”
The reason this, in particular, is compelling is that I’ve worked with some highly intelligent and capable people that were absolutely interdisciplinary and versatile in my 15 years of professional experience. However, I’ve never worked next to (as far as I know) a physicist. Now, I understand this is not a call to all physicists to abandon their pursuits to come solve big data challenges, but I don’t believe that business, in general, has embraced the fact that classic scientific brains and methods are needed when attacking big data objectives and challenges.
Maybe the next personal development task I take on will be to enroll in a Physics 101 class.
This article was republished from Brian’s blog, Centric Consulting – Boston
