by Angela Guess
Derrick Harris recently argued that Big Data and security are technology soulmates. Harris writes, “Sourcefire is now monitoring 2 million endpoints as part of its Immunet anti-malware product, and Hadoop is doing the heavy lifting of analyzing the hundreds of terabytes of data those endpoints are pumping into the company’s centralized data store. This just goes to prove my point that security and big data are a match made in heaven.”
He continues, “As is the case with many security services, Immunet isn’t a Hadoop product as much as it is a product that uses Hadoop (a sign that the technology is maturing). Two millions endpoints generate a lot of data (in number of items, at least, if not in volume), and it takes some special tools to store and process all that information. The more information that SourceFire can determine about threats, the better it can protect is users. Users don’t care about Hadoop, NoSQL or any other IT buzzwords — they just care that their computers are safe — but companies like Sourcefire certainly do.”
Harris adds, “According to an email from Zulfikar Ramzan, Sourcefire’s chief scientist within its Cloud Technology Group, ‘Hadoop is one of the more prominent technologies we use, though we have also built some custom technologies as well.’ He said the company also constantly evaluates new technologies such as NoSQL databases, but the trick is finding tools that are ‘well baked enough for use in production environments’ and that meet specific needs. Hadoop, for example, is great for general data mining purposes, but SourceFire has custom-built some tools for real-time workloads that the batch-oriented Hadoop can’t readily address.”
photo credit: LeonArts.at
