by Angela Guess
Sooraj Shah reports in Computing, “Hadoop and Spark are leading the way as the primary big data processing platforms for organisations in the UK, according to research from Computing. More than 500 people who work in IT responded to a nationwide online quantitative study among companies that have 100 or more employees from different sectors. Respondents included CTOs, CIOs, COOs, CEOs, IT managers, developers, as well as many others. While Apache Hadoop has become the de-facto big data storage engine, there has been talk of it being displaced for some processing tasks by newer technologies such as Apache Spark. However, the research still gives Hadoop a substantial lead.”
Shah goes on, “When asked which big data processing platforms the respondents believed their company would be using as their primary tool in 18 months, the biggest proportion of those companies who said they would be processing big data said it would be Hadoop (59 per cent), followed by Spark (17 per cent). Kinesis (seven per cent), Storm (four per cent) and Flink (two per cent) were other platforms on the list that respondents said they would be using, while more than a quarter (26 per cent) said that they will use ‘other’ big data processing platforms. Computing’s research also found that ‘advanced’ organisations – those businesses that are leaders when it comes to adopting and using technology to drive change – are relatively more likely to be using Spark as their primary platform, suggesting that it is catching up with Hadoop.”
photo credit: Flickr/ aotaro