Big data technologies is a great opportunity for companies to tap in to a pool of information that can help them create new services, improve customer experience and ultimately, increase revenue. But getting those results could require some companies to boost the talent pool within their IT organizations.

Data science, database management and predictive analysis skills represent just a few areas of expertise needed to fully realize the benefits of big data. Today IT and business leaders realize undertaking big data initiatives will also mandate they replenish the skill sets in their organization either by training existing staff or seeking out new talent for hire.

In search of skills

According to a recent research report, 57% of 1,000 senior and technology managers in IT intend to retrain existing staff, and 47% anticipate hiring new staff to address their big data needs. Because it is early days for many big data projects, one challenge for companies is finding the in-house talent that knows the right questions to ask when kicking off a big data project. The point is to pull meaningful business information from volumes of data, and that isn’t as easy as it might sound.

“Having the technical skills to know what it is you want from the data and what questions you want the data to answer is challenging,” says Ben Daubney, research consultant at technology research firm Vanson Bourne. “It could be one of the factors driving separate big data projects across different departments within one organization, different departments have different needs. Technical teams are trying to mine meaningful data, but it’s not necessarily known what insights are needed across a whole organization.”

The role of data scientist has become synonymous with big data implementations. Data scientists can couple knowledge of the business with statistics and mathematics to try to apply insights relative to the organization to the big data collected. Still, having a data scientist in house that understands the business and can apply meaning to the big data collected across an organization.

How do you Hadoop?

Another challenge introduced with big data is the variety of technology distributions around Hadoop. If Apache’s Hadoop—or a variant of it—is used, IT leaders will want to get in-house staff trained on the open source framework. But if using a vendor distribution for support or other reasons, the applications will differ and require updated training.  As environments mature, the technology used could also change, meaning IT professionals will have to continuously update their skill sets.

“Hadoop is not a single tool. It’s a stack of software tools with supported distributions. It’s a highly complex environment with a lot tools involved so how you use the tool, how you manage the tool, the immaturity of the tools or just the lack of tools will challenge skill sets,” says David Hodgson, Business Unit Executive for the new Data Management group at CA Technologies.

It is likely that a new big data project will reveal skills gaps across IT organizations, but identifying the specific set of challenges every company will face isn’t simple. Depending on the talent in-house, the skills lacking will differ across IT organizations. IT and business leaders need to inventory their current talent pool and align new skills with the business drivers for their big data initiatives.

The post Will big data reveal IT skills gaps in your organization? appeared first on Highlight.