Challenged with limited personal time and virtually unlimited information sources, the knowledge worker of today is becoming less efficient. Finding what we need, in the right context at the right time is increasingly difficult with current methods in web or enterprise search. In this article I’ll outline some primary issues that we face and how we can resolve these issues with new approaches that use semantic technologies. I’ll explain what semantic agents are, what they do and describe how they can help computers help us more effectively.
The End of the Information Age
The Web is the great aggregator, connecting over a billion keyboards to over a billion computer screens. Since 1989 the aggregation of information has enabled us to search for almost anyone, research and procure almost anything within seconds.
However the Information Age which we entered in the nineties and continues now maybe drawing to an end by the weight of it’s own virtues and success. We have been busy collecting information but haven’t been keeping it tidy. For us to be able to keep using it effectively our current management and retrieval methods need to be replaced by something new.
The Internet is home to countless information repositories. Each one is organized differently, categorized differently and each functions independently – each has no knowledge of its peers. Indexes of the Web are able to locate information across repositories and present it to us, but context and meaning of information with respect to its source is not articulated.
The success of the Information Revolution relies on intelligent workers that tirelessly analyze, organize and deduce knowledge from all these disparate sources…those workers have until now been solely human. We are facing difficulty as the repositories grow in size and number and we are now becoming our own bottleneck.
When you think about it, what do the mainstream search engines actually do? Not much, but very quickly. They scour information that only a human can understand, try to guess how a human would catalog it and then aggregate it in a private database. When we are looking for something the actual searching, comprehension and knowledge extraction up to you. It might always take a fraction of a second to find a piece of information, but, as the number and complexity of information repositories grow, locating the right piece and the knowledge it provides will continue to take exponentially more of our time.
Unfortunately in today’s scenario the case that information equals knowledge doesn’t hold up. Information doesn’t equal knowledge, and never can will the tools at hand.
The Knowledge Revolution
The advent of the semantic web has provided a framework for information contextualization, categorization and disambiguation. With the help of humans at the beginning of the process computers can now use logic and data interoperability to do a much better job of aggregating reliable information effectively. Semantic indexes of information are highly interoperable and context aware, allowing us to extract knowledge with far greater efficacy and speed.
However, information is also in constant flux. It continually emerges, changes and disappears. A posting on Craigslist.com will be at the top of its category for merely seconds, before it becomes buried under thousands of subsequent posts. Very soon it’s out of our reach. We may search for something on Monday, but the perfect match is posted on Tuesday. And when we perform another search on Wednesday it’s already too late.
The application of semantic search helps find information in the right context, but does not assist us with the temporal nature of information. We still have to manually, and routinely perform the searches ourselves. As increasingly more information is available to us this too becomes unsustainable.
If we are to truly optimize the way we find information and gain knowledge we must stop trying to find it. Why not let computers find it for us and tell us when they have. This is where autonomous semantic agents will enable The Knowledge Revolution.
Imagine we take a standard semantic search query and give it a life of it’s own. We send it out on a mission to find stuff for us. When it’s found something it sends us a personal message saying “Hey!, I just found something that’s a pretty good match, check it out when you have time.” We no longer have to be pinned to the search button, as a living query will continually look for new and changed information until we tell it to stop.
While out there it will provide us instant updates through any communication channel (email, SMS, voice) when it has found a match. And as it’s working continuously and responds almost instantly, the temporal nature of information is no longer an issue.
The vision is to let computers do what they do best… facilitate the decision making process for us, but not make the decisions. Present us with qualified choices and let us decide. The processing ability of humans cannot scale to support the knowledge revolution; only computers can do that. Semantic agents that act as our private and personal knowledge workers are instrumental to its emergence.
Agents will provide us sharper, timely, reliable knowledge with less ambiguity. So we can make the best decisions possible, and still be productive. In fact, agents are crucial to the continued increases in productivity.
Agents are in use today. With the development of semantic web standards, tools and computer processing power semantic agents are now a reality in real-world applications. Semantic queries can be encapsulated as agents and then freely distributed in special semantic repositories where they interact and seek each other out. When the right agents meet, and a match is found, their respective owners are notified.
The architectural framework for this type of interaction is very different from that of current search engines. In this higher functioning search, many of the key processes are decoupled, distributed and there is also a tunable interplay between efficacy and response time. The most appealing architectural feature is that we humans are finally decoupled from the computer as well.
As we can walk away and let the agent do its work, the immediacy of a response is not so important. A mediocre result in 0.8 seconds is nothing to an excellent result sent directly to us a couple of minutes later. Also, by using iterative filtering techniques, horizontal and vertical scaling of hardware and agent replication the model scales very effectively.
Agents can represent anything semantically, and have different roles and functions. They can represent people, products, knowledge, events and services to name a few. They only contain semantic meta-information about their owner, but include a reference to the owner’s location. They follow rules and criteria defined by the owner so as to respond with the right match in the right situation. The agent’s mission can also be updated while active, if so required.
There are many potentially useful applications; real-time information exchanges, product procurement, public safety, social networking, academic research & collaboration and media-entertainment.
The following case study outlines how agents are used to help people and businesses connect for resource, service and product needs on a continual basis.
Bintro is a business networking tool and information exchange that helps individuals and businesses increase performance and productivity. It introduces business professionals with specific needs and interests to other individuals and organizations that can match. It enables members to automatically find the most relevant knowledge, resources and solutions.
For larger enterprises Bintro serves as an effective knowledge management and collaboration tool that cuts across departments and functional areas. It helps people the information and answers they need as soon as they are available to solve their most pressing business challenges without sacrificing their own productivity in the process.
Bintro’s community members are in control of their information. You are matched with others based on the merits of your experience and skills. Because an agent represents you, you’re personal information is not revealed to your match until you choose it to be.
How it Works
After registration, new Bintro members are able to complete their profile and also submit broadcasts. Broadcasts are active notifications of something you are providing or something you need. For example you may provide financial consulting, employment or organically grown produce in New Jersey. You many need a nanny or babysitter (the semantics are not important to the system), a part-time job, or a specialized tax attorney.
You describe your broadcast in detail and set a duration for your search. The system then analyses your broadcast via semantic and NLP methods and creates an agent representation of it using RDF. It is then encapsulates it and sends it off to work for you.
Profiles are very similar to broadcasts, however they are more passive. Profiles contain information about your line of work, your research, skills, passions, hobbies and accomplishments for example. The profile agent then goes out and tries to match you with other members with similar things in common; this allows you to grow a personal network without having to post your personal details to everyone.
The agents continually interact and notify you of potential matches via multiple communication channels. It has done much of the time consuming work for you and now leaves the salient decision making up to you.
Whether it’s a profile match or a broadcast match or a combination of both, you can selectively chose who gets to meet you by reviewing summary information about your match and communicating anonymously though the site. When you feel comfortable you can then open your profile for review and start working together.
Bintro is fundamentally three different systems. The Bintro web application publishes agents and consumes matches. Message queues route agents and manage load allowing the system to buffer throughput and also scale effectively. Active semantic repositories store the agents, manage them and let them interact. Agents can be replicated between stores that are geographically dispersed and the stores themselves can be specialized for specific types of agents or agent requests.
When agents find a match or group of matches the results are sent back to the respective queue and back to the Bintro website. The whole process occurs within seconds whenever a match is found. The system allows agents to be updated if the member’s criteria change. Such criteria might be geographic distance, agent sensitivity to the criteria or addition or removal of information.
By using semantic agents Bintro is able to act as a great facilitator that effectively relieves its members from time consuming and tedious decisions and allows them to focus on decisions that will greatly accelerate their personal and business development.
Let’s Make Better Decisions
The practical implementation of semantic agents will vary from organization to organization. And their requirements and abilities will differ on a case-by-case basis. Standards are emerging that will hopefully consolidate agent’s interoperability and the environments in which they interact. The adoption of semantic agents will be driven by necessity; as they and their benefits become better understood the knowledge revolution will gain momentum.
We need greater help from computers to help us accumulate and consolidate knowledge across all disciplines and topics. We have the capacity to absorb vast amounts of knowledge, but limited tools to do it. We often make decisions that are based on limited knowledge, and feel satisfied with the decision not knowing the factors we didn’t consider.
Knowledge is power they say. I disagree. Information is power. Knowledge is wisdom.