by Angela Guess
A new article discusses the history of Hadoop and why it is taking over as the option of choice among start-ups. The article begins, “Even a modestly successful startup has a user base comparable in population to nation-states. The resultant mass of user data creates problems and opportunities… Until just a year or two ago, analyzing this scale of data required the same kind of enterprise solutions that LAMP [Linux, Apache, MySQL, PHP/Python] was created to avoid. Multiyear, multimillion dollar deals with the likes of IBM, Oracle, and Teradata. Of course, almost no startup can afford that kind of expense. Furthermore, the closed-source technological pedigree of these solutions makes them incompatible with startup engineering knowledge and culture.”
It continues, “Enter Hadoop. Hadoop solves these data processing problems in a way that is both startup-compatible, and technologically superior. As an open-source project developed by and for engineers, it’s very practical and squarely in the mainstream of startup engineering practice. And its architecture of map-reducing across of a cluster of commodity nodes is more flexible and cost effective than traditional data warehouses. Given the pent-up demand, it’s no surprise then that Hadoop is blowing up. To see how an open-source project is doing, the first thing you do is look at the developer mailing list traffic. If you gather all the Hadoop-related mailing lists and plot the number of messages, you get the classic hockey stick growth curve.”
The article goes on, “At the high-level, there are 3 areas where Hadoop is finding application in today’s startups: 1) Analysis of customer behavior, 2) Powering new user-facing features, and 3) Enabling entire new lines of business that were previously out of reach. Eventbrite is an example where Hadoop is powering new user-facing features. The rapidly-growing events service lets organizers manage and promote their events, and helps users find relevant events to attend. To help grow the company faster, their data services team is using Hadoop to feed intelligence in, under the hood, into various parts of the product.
photo credit: Hadoop
