Data intensity: The key to a data-driven future

What does the data-driven future look like? It will consist of systems that are:

• Highly automated and use data to make trusted, fair, split-second decisions.
• Personalized and situationally aware to cater to user needs.
• Able to address data movement, geographic distribution, governance, privacy, and security; and
• Decentralized, address data ownership and work in tandem with centralized systems to allow sharing of data for the greater good.

But we don’t need to wait for all of that.

The data-driven future is already here. 

An autonomous vehicle is an intensely data-driven system, sensing in real-time its environment and translating that into vehicle operations. At a level below autonomy, assistive technologies are also data-driven, relying on real-time data to produce insight i.e., the blind-spot detection system sends an alert or to make decisions about when to employ anti-lock brakes and crash avoidance systems. Successfully enabling such applications and use cases to be more data-driven is a journey that requires addressing complexity and adopting new approaches that enable you to better manage systems through maturity and sophistication. To assess digital maturity and resilience and level up your data-driven business, think in terms of data intensity. Data intensity is multi-variable and changes sharply as you move in more than one dimension. The data-intensity of an application depends on data volume, query complexity, query latency, data ingest speed and user concurrency. Additional dimensions might include hybrid workloads (transactional and analytics), multi-modal analytics (operational analytics, machine learning, search, batch and real-time), elasticity, data movement requirements and so on.

Data intensity is increasing

Data intensity isn’t just about data volume, it’s about what you do with your data. However, as data volumes increase, intensity grows. The intensity ramps up exponentially when the data also comes faster, creating the need for an application to handle 10 times more users while meeting the same (or better) latency SLAs. Intensity also increases sharply when the analysis of operational data in real-time combines with natural language interaction and recommendations. We live in a data-intensive era, and intensity is growing as organizations increase their reliance on data to better understand their customers and shape experiences. How your organization responds in the data-intensive era can either add more complexity and friction for you and your customers or it can provide you with new opportunities for differentiation and growth. Choosing an approach that leads to greater complexity and friction is clearly counterproductive. Yet historically, many organizations have worked from the assumption that different workloads require different architectures and technologies, and that transactional and analytical workloads must be separate. Managing data intensity in this environment creates inherent complexity, friction and data movement that adds latency and works against real-time insights. Fortunately, you now have the chance to revisit and challenge traditional assumptions to embrace, enable and get the greatest benefit from the data-intensive era. You can leverage cloud computing, which delivers unprecedented scale and flexibility and the opportunity for organizations to innovate and experiment; separation of storage and compute, which disentangles storage and compute requirements; and modern solutions that combine transactional and analytical workloads in a single engine for all workloads. In a data-driven organization, the day-to-day business operations, analytic insights from the operations and customer experiences become one in real time. That is intense: data intense.