Open Source Analytics Penetrating Deeper into the Internet of Things

By on

Click here to learn more about James Kobielus.

The Internet of Things (IoT) is evolving into globe-spanning utility that incorporates open-source componentry down to its lowest level.

The open-sourcing of IoT is a well-established trend, as amply attested by this and other sites. There is a dizzying range of open-source IoT industry initiatives.

However, I hadn’t realized how deeply open source analytics initiatives were penetrating the IoT until just recently, while absorbing the sessions at IBM InterConnect 2016. Chief among themes at this event were the convergence of open-source analytics and the IoT, many of which revolved around the themes of open analytics and cognitive computing in the IoT.

As I followed the discussions at that conference, it became clear to me that open-source data analytics is beginning to pervade the IoT at many levels. Open-source efforts are disrupting the entire IoT, and are not limited to one vendor’s solutions and partnerships in this space. The levels on which this trend is developing are as follows:

  • Hardware: Open-source hardware is starting to come to the IoT in a big way, though, as this recent article attests, it’s actually a longstanding practice in high-tech, stretching back even before the “homebrew computing” days of the early PC industry. These days, there are diverse open-source IoT hardware initiatives, including Openpicus, Sun SPOT, PowWow, BeagleBone, Intel Galileo, Pinoccio, WeIO, and WIZnet. In addition, general purpose open-source hardware platforms—such as Raspberry Pi and Arduino—are increasingly being deployed within IoT edge devices. What caught my attention at InterConnect 2015, was the statement by Harriet Green, IBM general manager for Watson IoT, of their partnership with the Raspberry Pi Foundation to enable Watson cognitive analytics to execute on IoT endpoints that incorporate that open-source hardware technology.
  • Analytic algorithms: Open-source analytics will be embedded in every component of the IoT, from gateways all the way out to every edge device. I commented a few weeks ago on various industry initiatives in that regard, including efforts involving AMPLab and Google. Then, in mid-February, IBM announced that it is submitting to the Apache Software Foundation a development tool, called Quarks, for embedding in-memory cognitive analytics software in myriad IoT platforms. As I discussed in this blog, Quarks can manage and analyze continuous streaming data on any IoT device. It provides a single runtime for analyzing IoT data at the edge. It can run on IoT edge devices and gateways, and enables continuous correlation of data across the IoT. And Quarks, now on GitHub, works with Spark, Hadoop, and many other data and streaming analytics environments.
  • Composable microservices: I’ve recently blogged on the emerging practice of building analytics as composable microservices that execute on the cloud, including the radically distributed “fog computing” environments that support the IoT. This microservices approach is of longstanding, tracing its lineage back in part to the “service-oriented architecture” paradigm that was in vogue in the past decade. At the most recent InterConnect, IBM announced that it is unveiling an open-source software tool for event-driven programming of data analytics, such as cognitive IoT, as composable microservices. The newly announced OpenWhisk lets developers build more feature-rich event-driven apps quickly and easily. IBM’s intent in open-sourcing of this technology is to generate a powerful ecosystem of event providers and consumers to develop the platform. Developers can use OpenWhisk to quickly build microservices that automatically execute cognitive analytics and other IoT software code at the IoT edge device in response to events such as receiving sensor data.
  • Developer collaborative ecosystems: I’ve recently commented on the need for open-source collaboration environments that enable distributed teams of IoT app developers to tap deep libraries of algorithms, data, and other project artifacts through open APIs. At InterConnect 2016, IBM and GitHub announced an open-source repository more than 140 APIs and services for cognitive and IoT app development, as well as new sources of data.  The initiative will enable developers with the ability to view, collaborate, integrate APIs, iterate, deploy and update on one single platform. This significance of this effort relies in the fact that GitHub, the largest host of source code in the world, offers free repository services that are used in numerous open-source software collaborations.

These trends are inexorably driving the IoT to ubiquitous adoption in consumer, business, industrial, and other uses. It’s driving the standardization, commoditization, and adoption of low-cost and free IoT technology. It’s accelerating the adoption of this technology in a growing range of innovative solutions. And, through the embedding of open-source cognitive technology into every IoT endpoint, it’s fostering a world where every device will be capable of going fully “autonomous”—in other words, able to sense, act, and react intelligently and dynamically to changing environmental circumstances.

In other words, open-source IoT technology is transforming our world into a place where everything can truly think for itself.

Leave a Reply