Hadoop is not a stranger. Most of us at least know that it’s a big data framework, and that it has a lot of potential. A 2015 study by Gartner shows that 26% of responders have already started using Hadoop in some way, while 11% are planning to adopt it within the next year and 7% within the next two years.
While the Gartner study pointed out that a majority of responders (57%) have no current plans to integrate Hadoop, there are very good reasons for this: a lack of Hadoop experts (cited by 57%) and a lack of insight into the benefits of this technology (cited by 45% of responders).
Hadoop vendors are attempting to address the skills gap by ramping up their training programs. But this leaves an important question for business users: What’s the value of Hadoop these days?
How Businesses Are Getting Value from Hadoop
It’s easy to think that Hadoop – and indeed, big data analytics as a whole – is more or less the exclusive province of mega-companies with strong online and e-commerce presences. But that’s not the case, Michele Nemschoff of MapR Technologies writes in this article. Hadoop is being integrated into organizations of all kinds and sizes. Among the examples she cites, we can find such diverse fields as financial services, transportation businesses, and power grids. In each case, using a Hadoop-based data analytics system can save money, predict maintenance needs and usage spikes, and even improve fuel efficiency.
Because Hadoop is intrinsically flexible and scalable, this has made it a viable big data framework for growing companies. But since data analytics is anything but a static field, businesses must invest in a robust Hadoop framework, one that can support the growing multitude of technologies and data types. Only then will their big data setup be able to give them the results they need: a better understanding of the entire organization, garnered from a flow of real-time information. This understanding can then be leveraged into intelligent decisions.
Using Hadoop on New Kinds of Data
Let’s revisit the idea of new data types for a moment, because this is one area where Hadoop really shows value. For example, consider clickstream data, which shows where and in what order a person clicks a mouse. On a website, this data can give you incredible insight into how effective your tactics are. With Hadoop, you can not only analyze and store the data, you can also join it with data from other sources for an even better understanding of audience behavior.
The same goes for other relatively new data types, like sentiment data. Sentiment data from a social media account needs to be joined with other data types to give it the necessary perspective. This includes figuring out the scale of the reaction and which segments of the population agree with it.
Server log data has presented a different problem for analysts: there’s so much of it that it can be hard to manage. Since server data is often critical to understanding security problems, Hadoop’s capabilities in handling large amounts of data make it especially important.
How Can You Get the Most Value From Hadoop?
Let’s bring this all home to the business level. Hadoop can process large amounts of information, and it can do it efficiently and at a lower cost than some other frameworks. But to get value out of Hadoop, there are things you need to do – and things you need to avoid.
First of all, you need to avoid setting Hadoop up for tasks that it wasn’t designed to do. Cloudera’s Mike Olson notes this about the limitations of Hadoop: “It was never designed for high-performance transactions. Applications that demand that level of integrity and consistency ought to run on a different architecture.”
Secondly, to get the maximum benefit from Hadoop, you need to pair it with the right people and the right complementary tools. The right people – the ones with that much-in-demand Hadoop expertise – can be nurtured or recruited from inside or outside your organization. The right tools are those that take data and turn it into information. These are analytics tools (for analysts) or data visualization dashboards (for business users).
As data continues to arrive with greater volume and variety, it’s only logical to expect an increase of interest in Hadoop and other big data frameworks. Companies who beat the rush and start integrating this technology now will be in a better position to meet the demands of the future.
