Yesterday The Semantic Web Blog discussed how personalized mobile assistance came up on the lists of a bright future in the eyes of semantic web experts (see here). Sharing that vision is the team at Vital.AI, the NYC-startup founded by Marc Hadfield. Its Thrive.AI app, also a contender at SemTech’s Startup Competition, is a personalized semantic shopping agent for the iPad, but the underlying Vital.AI platform on which it is built provides an integrated suite of components for a variety of knowledge-centric, intelligence-rich, Big Data-driven applications.
The e-commerce agent, Hadfield told attendees at SemTech in San Francisco last week, was the company’s own foray into figuring out what it needed to add to the platform to make it easier to build apps that bring semantic technologies and Big Data together.
The platform is designed so that applications built upon it need care only about data and the user interface, leaving the infrastructure – components from natural language processing, to a Frame Logic Rule and inference system, to Interest Graph analytics using Scala, to an Allegrograph Triplestore with Groovy Interface, to data analysis via Hadoop — to Vital.AI, tweaked as necessary to the app’s needs. “The problem of Big Data in semantic technology applications is that you have to do a lot of roll-your-own stuff,” Hadfield says. “Here is one uniform data model driven by an OWL ontology at the top level.” The ontology provides the framework for sharing RDF-encoded data across all subsystems, including delivery to the IPad.
The system scales out by distributing work over many servers in a message queue, with data arranged in process flows and coordinated across many different processes. “Each process engine could be doing NLP or semantic inferencing or graph analysis. Say it’s the NLP component – we could run ten of those and just be ten times faster in terms of processing data,” he said.
Semantic application agents, he noted, are based on two main features: a user interface that can anticipate what a user needs, and the ability to construct and execute a plan to meet those needs. The motive for Thrive.AI is product discovery, closely leveraging users’ data in a trustworthy way, from what they Like to what they tweet about to what’s in their email to what’s on their calendar. “Maybe you get a lot of Gilt emails – this reads them, figures out what is on sale that you might like,” he says. Or, if a user’s calendar has a beach trip coming up, it could start suggesting purchases appropriate to that event, as well as to the individual user’s tastes, drawn from analyzing and extracting data entities from what can be very large graphs of interests and social connections, and then extracting appropriate product details from multiple sources such as Amazon and Daily Deals, then categorizing them by certain dimensions – a product genome, so to speak, for good values, luxury brands, and so on. “The consumer interest graph essentially is the DNA for our user….Like Pandora has a music DNA, this is the shopping DNA,” he explained.
Meeting those shopping goals is the work of the agent planning module, and a scoring metric for ranking items effectively that’s based on reviews, as well as prior knowledge of brand profiles. And it also tailors results to what the user cares about, based on what the system learns from user feedback, such as that a user looks only at high-quality goods. “It’s very similar to Zite or Flipboard [magazine] applications, but the idea is to have nice experience to look at products and drill down on the iPad through the agent interface.”
But e-commerce is just one app where semantic technology can meet big data and help the user get to what he wants. Hadfield also mentioned the applicability of the underlying Vital.AI platform for general content parsing, to deliver news and other stories personalized to users, and even opportunities in the personal health care management space. He said that other customers are using the platform now to explore other applications.