Semantic Annotation Comes To Home Automation

By on
Courtesy: Flickr/ Moyan_Brenn
Courtesy: Flickr/ Moyan_Brenn

by Jennifer Zaino

Everyone knows The Clapper for turning electric equipment on and off, right? Sing along: “Clap-on, clap-off….The Clapper.”

Things have come a long way since then, with security, energy management, and more coming along to help turn the average house into a smarter home. Now comes a chance to take things to another level, with semantic-based resource discovery and orchestration in home and building automation. Research led by Michele Ruta, assistant professor at Technical University of Bari, takes on the challenge of bringing together the worlds of semantic web and automation in order to improve what Ruta says are very poor user interaction scenarios.

How? In home automation solutions today, he says, the user is limited to very basic scenarios and very static interaction that requires pre-programming the capabilities the home can assume. “It should be possible to have dynamic interaction, more intelligent interaction with the user, and decisions should be done according to user interest, to a user’s profile,” Ruta says. Semantic technology can be called upon to annotate users’ profiles, interests, and needs against home automation profile options, and make the match between them, he says.

“The final goal is to reduce the semantic distance between the user and home profile by turning on and off devices, by setting appliance characteristics and so on,” he explains, adapting the home based on an understanding of the user’s specific situation. A mentally tired user, for instance, may get gentle music playing low in the background and a cozily warm temperature setting – all of it happening automatically, without the user having to engage in any direct interaction with a home automation system.

The research aims to enable this by leveraging a crawler on a user’s smart phone that that uses machine learning and data mining techniques to extract a user’s profile considering the person’s real life experiences that day – for instance, recording GPS traces for the user that day to establish her spending most of the day at a location that is equated with her office, and matching that up with the current time to determine that she is likely tired after a day of work and in the evening hours, he says. This annotation is all open to correction by the user, or she may just input to the smartphone agent her condition to start with.

Devices in the home, like ovens, will be annotated in the factory with predefined profiles that can be enriched for central home agent technology. Sensors for things like temperature, for instance, can contribute that information to the home agent, too.  (The thinking also involves having a device that generates specific requests for other devices in the home – for instance, it can ask them to assume a given setting, or to turn off while it conducts a high priority task if that will help it undertake the job more efficiently.)

“Basically our research is devoted to the algorithmic calculation of the semantic distance [between the user condition and the home setting],” he says. The work includes the development of an ontology registering the domain conceptualization, that is the vocabulary for the annotations; and the knowledge base with device characteristics and user profiles and appliance general features. The inference services named concept abduction and concept contraction calculate the semantic distance between the user and home profile, and allow to possibly correct any conflicts there. “The final goal of this inference task is to reach the best profile of the home with respect to the user profile,” he says. “The semantic distance explanation is useful for doing this because it lets us regulate the home settings in order to satisfy to the best extent the user needs.”

The work takes into account staying compatible with accepted home standards, and the project is associated with the KNX Association, whose technology is a standard around home and building control. Ruta says he and his team have developed a simple prototype house in the lab leveraging its research.

The next steps for the effort involve increasing the expressiveness of the logical formulas it uses for annotating and reasoning on the annotations and to move toward more complex networks to support similar work in industrial scenarios. “We also want to use this approach to control energy consumption,” he says, with devices transferring semantically annotated information about the energy they need and the home then calculating overall energy needs to negotiate correct energy profiles with providers, along the smart grid general framework.


We use technologies such as cookies to understand how you use our site and to provide a better user experience. This includes personalizing content, using analytics and improving site operations. We may share your information about your use of our site with third parties in accordance with our Privacy Policy. You can change your cookie settings as described here at any time, but parts of our site may not function correctly without them. By continuing to use our site, you agree that we can save cookies on your device, unless you have disabled cookies.
I Accept