Addressing the Big Data Talent Shortage

By on

by Gil Allouche

Big data is growing at an enormous rate. There’s more data than ever, and there are more industries beginning to institute big data technology too. Each day there emerge more and more ways in which big data can positively affect our lives and increase effectiveness and efficiency. It’s also becoming easier and easier to access big data due to big data in the cloud. Big data in the cloud eliminates the enormous startup costs that companies with on-premise infrastructure incur. It also takes mere days to get going, instead of weeks or months like on-premise infrastructure. All the emergent technologies and increased visibility are coming together during a period of extreme growth in the industry.

That huge growth has caused an increasing call for talented data scientists, data executives, software developers and other data and tech oriented positions. There are openings all over the place, but unfortunately, there is an enormous shortage — there’s nowhere near enough supply to meet the demand. Something about this has to be done if big data wants to continue to grow at its unprecedented levels.

On the one hand, it’s not too surprising that there’s such a challenge in finding qualified individuals for the big data industry. The tech industries, especially in the information technology and information system sectors have faced the challenge of meeting the job demand for quite some time without complete success. They have been two of the fastest growing industries for some time now and have been constantly inundated with technology trends from bring your own device to the Internet of Things. It seems that the tech industries are destined to constantly fight shortages. The big data field is no different. Fortunately, however, there are important steps being taken across the big data field to address the talent shortage and keep the adoption of big data growing at exponential levels.

One of the challenges to the shortage is finding people qualified for the different positions. It’s not just data scientists with PhD’s that are needed. They make up a big part of the crew, but there’s also a need for data executives, software and hardware pros and others positions. So, what is currently being done about the shortage? And, what additional steps can be taken to solve these problems?

An important step, and one of the first ones to be taken, has been to invest in graduate programs across the country, many of which are at business schools. The business school aspect is important because it’s preparing executives who will not only understand how to run a big data team, but they’ll also understand how a big data platform works and the analytical side to it. They’re doubly prepared. MIT is one of the schools that has been taking steps forward with big data. They recently opened up an online big data course and had more than 1,000 people per day sign up for it.

But, it’s not just grad students that are interested in big data, nor are they the only ones that can make an impact on the industry. Increasingly, universities are implementing data science, analytics and big data business programs at the undergrad level. It’s giving students a chance to get involved in the big data process before they get to the master’s level. There are plenty of opportunities for entry-level professionals without a masters or PhD.

Beyond the college level, companies need to take big data education to high school and earlier. As fast as big data technology is growing, it’s still an unknown to huge pockets of the country and world. If kids don’t find out the career possibilities of big data until college, it may be too late. With an increased effort to market the possibilities of big data to younger individuals, the big data sector can vastly increase its chances of meeting the needs. It’s important for these kids to understand that the data science skills required are extremely useful no matter the field they choose to enter.

Sure, there’s a definite shortage at the moment and will be for some time to come. However, with a focused effort, interested parties in big data can work together to educate more and more of tomorrow’s big data professionals and eliminate the shortage and secure big data’s world-changing influence.


We use technologies such as cookies to understand how you use our site and to provide a better user experience. This includes personalizing content, using analytics and improving site operations. We may share your information about your use of our site with third parties in accordance with our Privacy Policy. You can change your cookie settings as described here at any time, but parts of our site may not function correctly without them. By continuing to use our site, you agree that we can save cookies on your device, unless you have disabled cookies.
I Accept