Assistance, Not Resistance: 11 Keys to Big Data Analytics Success – Part One

Large organizations continuously seek ways to leverage existing data in new architectures to improve business aspects—including anything from cost-reduction initiatives such as consolidating systems; construction of predictive machine learning algorithms for analytics; or even tried-and-true data quality, reporting, and other types of analysis. Once these core capabilities are developed, it becomes easy to extract valuable insights to improve the bottom line.

Still, many utilities indicate scarce, patchy data as a major pain point which makes constructing big data analytics solutions difficult. And when you take into consideration that over 90% of today’s data has been generated within the past two years, the problem isn’t going anywhere. With data volumes growing by the day, collection, integration and management will only become more complex.

To make construction as seamless as possible, address these five key considerations for successful big data analytics solutions:

1. Analytics strategy and projects must be business driven

If business division leaders do not agree with the project or strategy, it will fail. Take the time to ensure that business leaders are fully supportive of the effort. Address all their concerns about analytics initiatives so that there is no resistance to making progress. Ensure assistance, not resistance.

2. Be agile

Various forms of agile software development methodology have become popular recently, and for good reason. Agile development allows for flexibility in scope and requirements, within a fixed timeframe, to ensure that something will be delivered, and will ideally include the most valuable functionality.

Project sprint cycles can include existing data discovery, as well as its accessibility and value. Incremental development allows organizations to prioritize and add the most valuable new data and functionality to the pipeline backlog, while also improving previously delivered products. However, be careful not to define a complete list of required functionalities at the start, declaring the project late until all the requirements are delivered. Agile means that the timeframe is fixed, but the scope is not. Do not just say you are agile, be agile.

3. Define a strategy for master data management

Especially in older businesses, such as utilities, record-keeping started within various divisions on paper and moved slowly to computers and databases. Those divisions created systems to handle the data that they cared about. Oftentimes, the same information is needed by multiple divisions; however, each division might keep their own records. Under these circumstances, a strategy is needed to decide which dataset is the most complete, merging other information into it to be used for analysis. Combining data from various sources might also mean that record identifiers are no longer unique, so it may be necessary to create new globally unique identifiers, and map those to each system’s local identifiers. Without this strategy, various ways might exist to compile reports and do analysis. If a different basis is used, the results can vary greatly, making the analysis platform seem untrustworthy.

4. Build a strategy and requirements for metadata

Combining data from various places into an analytics environment enables slicing and dicing of data that was not previously possible. However, it is important to ensure that results can be explained. That is why every piece of data in the analytics database must contain lineage, or pointers back to the original source system, as well as information about the data quality, business contacts, protection constraints, and any other important aspects. This means that a cataloging system must exist for referring to source companies, systems, environments, divisions, entities (tables), records, and attributes. It might not be necessary to have all this information available in the analytics environment, but it sure makes it nice!

5. Document existing systems and processes

Building an analytics platform without documentation about existing systems, data sources, tables, columns, and values is very difficult. Business processes and integrations between systems must be understood to know which source is the system of record. This information needs to be defined in order to ensure success. If it doesn’t exist, plan to build it as part of the analytics project and keep it up to date from that point forward—or it won’t be nearly as effective.

As with other large, enterprise-wide initiatives, analytics projects tend to face all sorts of unforeseen difficulties in accessing the data. Plan to spend a large amount of time discovering what exists, who understands it, and what the best source is for each topic. Start small, gain experience making incremental changes, and you’ll go far.

Watch for Part Two, with six more key considerations for big data analytics success.

Check out our website to discover how Xtensible Solutions makes analytics implementation more efficient and effective.