Real Time Communications Featured Article

DataTorrent RTS to Process Big Data in Real Time at a Billion Events per Second

June 05, 2014

The ability to crunch massive amounts of data allows organizations to process information in real-time instead of looking back at what happened. Whether it is flash trading to gain milliseconds, or trying to figure out what customers are thinking about a product in real-time, the ability to analyze information quickly, has many competitive advantages. DataTorrent has now upped the ante by being able to process more than a billion data events per second with its DataTorrent RTS platform.

This new solution is built on top of Hadoop 2.0, and it processes enormous amounts of data in-memory by crunching and correlating the information as it expands and detracts throughout the day by adapting to the conditions in real-time. Because the platform is fault-tolerant, it is resilient to disruptions that could potentially force a system down and data loss.

While many big data analytics service providers do crunch the information and provide valuable insight, latency is still a problem many organizations face. Looking back at an event after it happened is useful so the mistake doesn't get repeated, but what if the problem can be detected while it is happening so action can be taken to minimize the effect of the damage.

What DataTorrrent says it has done is allow organizations to see data as it is being generated so they can react to an event as it happens in real-time; the implications are huge to say the least. Imagine if GM could have stopped the assembly line when only one defective vehicle was detected. Innocent lives would have been saved, and the company wouldn't have to recall millions of cars. That is the potential of being able to process more than a billion data events per second.

DataTorrent can be used by any industry that needs to crunch large volumes of data, and they can build applications on top of the platform to process what they need. This includes automatic triggers so decisions can be made automatically or by people as soon as an incident takes place. The streaming platform fits on top of Hadoop 2.0 and it is compatible with Cloudera, MapR and other Hadoop distributions.

In today's digital customer centric world data is becoming a commodity as valuable as any other resource, and companies that are able to appreciate this information can use it to differentiate themselves from their competitors by leaps and bounds.

Edited by Stefania Viscusi

Article comments powered by Disqus

  Subscribe here for RTCW eNews