The technology you use to facilitate analytics adoption can make all the difference between success and failure.
What are the magic ingredients in the recipe for successful and prompt analytics adoption? In our last post, we talked about usability and scalability. We briefly mentioned building the right technical framework as part of the foundation for analytics. But what exactly is a good analytics framework?
That’s what we’ll look at in this post.
Introducing the Analytics ‘Skeleton’
Think about your skeleton. Without muscles, your body wouldn’t move. But without a skeleton holding it up, your body wouldn’t do much either. The same can be said for analytics. While performing data analyses, building data visualizations, and presenting the results to business users translate information into motion – serving as the muscle in this illustration – they need a robust framework to support all that movement and give it a structure on which to exist.
That, folks, is your analytics framework: the technological structure that makes analytics action possible. (In this analogy, I suppose we’ll liken data to the food and water that power the human body. But we’re not going to concern ourselves with data sources in this article. Let’s take it for granted that high-quality data sources have already been identified in the research and planning stages.)
So, what sort of ‘skeleton’ does your analytics system need?
4 Things a Successful Analytics Framework Must Have
The question of infrastructure should always be considered when you talk about an analytics framework. There are arguments for and against on-premises systems (with the ‘against’ being largely in terms of maintenance, expense, storage, and scalability issues) and Cloud systems. Once again, I will just add that many companies choose either Cloud-based or hybrid systems because of the innate flexibility and scalability of the Cloud. Plus, it’s often a more affordable route.
The exact choice of environment will depend on your organization. Whether your choice is on-prem, hybrid, or all Cloud, the important thing is that you should have a high-performing, robust, and secure system with enough flexibility to support your future needs. What kinds of technologies go into such a system? Let’s take a look:
Data Storage and Management
Data storage and management technologies include data warehouses, data lakes, and databases. A popular solution is to have a data warehouse and one or more specialized data lakes. (Essentially, data warehouses store structured (i.e. processed) data and data lakes store large amounts of structured and unstructured (raw) data. For a more detailed explanation, see this DataCamp article.) This approach saves on costs, as data lakes are less expensive while data warehouses make analysis more efficient.
Data lakes and warehouses are usually accompanied by relational databases and NoSQL databases. Relational databases are the familiar ones that store defined, structured data in tables, rows, and columns (much like an Excel spreadsheet does). NoSQL databases can store data as relational tables or in a much looser, text-based format, which allows for the storage of unstructured data.
Key goals for data storage and management are flexibility, scalability, and the ability to handle three of Big Data’s 4 Vs: velocity, variety, and volume
Data Processing and Basic Analytics
From storage, we move on to processing and basic analytics. In this stage, the framework must integrate data from different sources, process it (to make it usable), and classify it. Where data is being stored on multiple platforms and in multiple formats, stream analytics is very useful; it allows the system to filter, aggregate, and analyze incoming data. It’s also useful in connecting and integrating external data sources into the framework.
Key goals for data processing and analytics are velocity and variety (again) as well as integration – both in terms of varied data sources and types and in providing results that mesh with the rest of the system.
Basic data analytics functionalities are must-haves in the modern business world, but they are not ALL of the must-haves. Increasingly, businesses are turning to advanced (or enhanced) analytics, or analytics that has Artificial Intelligence capabilities built into the system. AI can be employed at various points in an analytics pathway; the most popular applications include using:
- AI to automate and improve data ingestion, storage, management, processing, and retrieval.
- Natural Language Processing (NLP, a subdiscipline of AI involved in recognizing human language) for voice search and text processing.
- Machine Learning (ML, a subdiscipline of AI) for predictive analytics and self-learning programs (among other things).
Adding various AI components allows analytics to be faster, more adaptive, and more productive. AI is adept at finding patterns and extracting insights that might otherwise be lost in a mountain of data; this gives end users information that’s more actionable and less prone to error. When trained properly on inclusive and high-quality data sets, AI can also reduce bias and incomplete results.
All of the above is never seen (and only indirectly experienced) by the business user. For them, the true test of analytics often revolves around how easy it is to access.
In this area, a good analytics framework will include presentation tools (e.g. dashboards, chatbots, and other interfaces that allow the user to interact with analytics systems); such tools usually have some data virtualization solutions as well. A user-friendly interface makes it easy for the business users to find the data they want; data visualization tools allow them to view it in an understandable way, e.g., through graphs and plots. This lets business users work with data independently, without constant recourse to the IT or data science team. And thus, the presentation layer plays a key role in the ultimate goal of data analytics: getting insights into people’s hands at speed.
With Analytical Frameworks, You Get What You Give
At this point, you can probably see why the framework of your analytics system is critical to its success; without it, your efforts are almost certain to collapse. This is one area where organizations absolutely must invest time and energy at the outset – either in-house or by partnering with an experienced analytics provider. But, like so many other things in life, the effort you spend in laying a good foundation will reward you in the future.