As The Semantic Web Blog discussed yesterday here, the Virtual Personal Assistant is getting more personal. Microsoft officially unveiled Cortana as part of the Windows Phone 8.1 smartphone software at its Build event yesterday, and the service effectively replaces the search function on Windows smartphones, both for the Internet and locally.
This statement served as the theme from corporate vice president and manager Joe Belfiore: “Cortana is the first truly personal digital assistant who learns about me and about the things that matter to me most and the people that matter to me most, that understands the Internet and is great at helping me get things done.”
The Bing-powered Cortana is launching in beta mode, and was still subject to a few hiccups during the presentation. For example, when Belfiore asked Cortana to give him the weather in Las Vegas, it reported the information in degrees, and was able to respond to his request to provide the same information in Celsius. But he couldn’t get her to make the calculations to Kelvin. But, he promised attendees, “Try it yourself because she is smart enough to tell you the answer in Kelvin.”
And Cortana did display those smarts in other respects. Belfiore explained, for example, that the personal assistant offers a notebook feature as “a transparent way of putting the user in control of the relationship.” The user can tell Cortana the interests and the people he’s most interested in to get served up the news and other information pertinent to those subjects, and Cortana also will ask or infer other interests and people that matter most to the user. “The more I use the search function, the more Cortana learns about me,” he said, and the service will ask or confirm if it’s right that those are the things or people the user really does want to track.
Users can give Cortana permission to take on other roles, too, such as reading all their emails in search of other things that warrant tracking. It, or “she,” as Belfiore referred to Cortana, “can recognize flight itineraries in email, and ask if she should she track that,” and once the user provides consent to that, Cortana can provide proactive notifications of time changes in flights, for instance, or prompts to leave for the airport.
Belfiore demonstrated a variety of other ways in which users can engage with the service using either spoken or typed natural language queries and commands. For example, he showed how Cortana can draw on Bing’s relationships with other services’ data feeds – such as Yelp restaurant reviews – to provide data on 4-star Mexican restaurants in Palo Alto in response to Belfiore’s request to find the best Mexican restaurants in that area. He was able to further ask the service to show which of these take reservations, though he hit a stumbling block when he asked Cortana to call the second of the two it had narrowed the choices down to, rather than explicitly naming the restaurant.
He harkened then to the fact that this is still in beta, as Microsoft continues to train the service “on the back end with lots of voice utterances to improve its speech recognition as things go on,” he said. Right now, the focus is working on helping Cortana learn a ton about sports, for example. “We also are inviting you all to help make her smarter,” Belfiore said. “When people start talking to their phones in natural language, we want to use the natural language capabilities in the cloud to make your apps work better,” he said.
As an example of how apps and Cortana can integrate together for better usability, Microsoft already has worked with Skype on a new version for Windows Phone 8.1 to recognize the contact a user requests to call. With Hulu, users can request to add shows to their queue in natural language: Cortana opens up the app, and the service recognizes the request to add the show. Its collaboration with Facebook lets users make a request to see what’s up with a particular friend. “It launches the app, the text is parsed in the cloud,…[the friend is found], and you can see his news feed,“ he said. The Windows Phone 8.1 release also gives developers the ability to let their users speak in natural language to future apps they create, as well.
Cortana also will work the people end of the personal assistant spectrum with capabilities such as tracking contacts and users’ interactions with them, and then providing the user with information about updates on those individuals that might be useful to acknowledge in future interactions with them – such as promotions they’ve earned. Or users can themselves remind Cortana to remind them about bringing up topics with their contacts the next time they interface with those people, whether in a phone call, chat or email. “That’s something other digital assistants don’t do,” Belfiore said.
In Other News
Windows Phone 8.1 and Cortana will come pre-installed on new phones starting this month, and rolling out to existing Windows smartphone users over the coming months. While these got the spotlight, Microsoft announced other news at the event as well, including the Bing Knowledge Widget that is designed to “empower every webmaster with the power of Bing’s knowledge repository,” which has hundreds of millions of entities and billions of relationships between them, according to the vendor.
Those sites that leverage it can enable their visitors to have quick access to information about the entities on the page – as images, links or images and links — from the knowledge repository simply by hovering over objects.