Sensage Blogs

These days it seems everyone wants to talk about life in “real-time”. Last week, the San Jose Mercury News, ran a piece called the “Real-Time Web, the valley’s new obsession” (http://www.mercurynews.com/business/ci_13342816). The main theme of the piece, “What’s fast enough?” is thought provoking. In particular, I loved the trendspotter debate about whether Twitter was “real-time” or “near real-time”.

My reaction to the article?  I high-fived my son at the breakfast table. You see, for those of us in the security and data warehousing world, it’s great to see a pervasive, general business debate about the value of real-time analysis — issues we’ve been dealing with for years. Particularly in security, we’ve been debating (or some may say splitting hairs) about “What’s fast enough?” for years. To us, the conversation isn’t new at all; it’s based on real technology and real needs. It is evolving however, from a focus on “real-time vs. near real-time” to one focused on “real-time and all-the-time”.

Why? Five main issues are driving this evolution:

1.  New battles. Speed and accuracy are essential when it comes to new battles such as cyber threats and monitoring core intellectual property. For instance, in markets like healthcare and finance, the cost of a missed security breach can extend to irreparable financial or reputation loss. In defense, it can mean lives are lost.

2.  New data. The key to responding to these new battles is event data — time-stamped, append-only data — and it’s the fastest growing data on the web.

3.  New requirements. Proper response to these battles starts with detecting threats from terabytes of events as they are occurring – in real-time. Proper response extends to analysis of years of collected event data. In some cases, this means sifting through hundreds of billions of records, to find fraud, criminal activity, or, simply, errors. In each case, the need for wickedly fast and complete response is essential.

4.  New technologies. No surprise, vendors are stepping up with solutions to address this large, new market. SIEM and log management firms have been at this for years. More recently, data warehouse firms are joining the fray: Teradata with the launch of their Extreme Data Appliance 1550 and Netezza with their Mantra Compliance Appliance. Set Google Alerts for event data, log management, or real-time analytics and you’ll see what I mean. The vendors are coming.

5.  New expectations. I think the Merc article nailed this one. Thanks in part to the “instant on availability” of social media, SalesForce.com, Zillow, and dozens of other SaaS apps -we’re quickly getting trained to expect results without time-out for manuals or training sessions. In short order, these same expectations will be “table stakes” for business intelligence and security applications.

Last week, I was at an industry gathering hosted by Sierra Ventures – their annual CIO Forum. A few things stood out. The first was the CIO’s peer discussion about delivering “real-time analytics” and self-service to their business users. The second was Joe Tucci’s keynote session about the next wave of IT. In his talk, he stressed the importance of speed and self-service enabled by cloud computing as being at the heart of the next wave of IT. He said the change will be bigger than anything we’ve seen, will give unprecedented power to users, and will have a huge mortality rate for those who can’t adapt.

This echoes what we’ve heard from our customers.  They’re telling us this new wave can’t hit “fast enough”. They’re in the game and deploying solutions to make this vision of “Real-time, All-the-Time” a reality.

permalink