by Angela Guess
Brian Proffitt of ITWorld reports, “Hadoop has been the undisputed king of big data for a while, but while the emperor has some clothes, the limitations of Hadoop have made the outfit skimpier than it needed to be. Now Hadoop developers are building the new processing system known as YARN, to knit the emperor some better clothes. YARN was recently upgraded to a full-fledged sub-project within Apache Hadoop. In the context of the Apache Software Foundation, that’s kind of a big deal: previously, YARN was worked on under the MapReduce sub-project, so moving it out to a separate project with its own focus is regarded in the Hadoop community as a significant step.”
Proffitt goes on, “Explaining YARN can be a tricky thing. If you recall, Hadoop is made up of two key elements: the Hadoop Distributed File System and the MapReduce processing engine. It’s MapReduce that gives Hadoop much of its mojo for processing big data. The Map part is accomplished by dividing computing jobs up into defined pieces and shifting those jobs out to the machine on the cluster where the needed data is stored. Once the query is run, that dynaset is Reduced back to the central node of the Hadoop cluster, combined with all the other dynasets from the cluster’s machines.”
