“There is nothing more difficult to plan, more doubtful of success, nor more dangerous to manage than the creation of a new order of things…. Whenever his enemies have the ability to attack the innovator, they do so with the passion of partisans, while the others defend him sluggishly, so that the innovator and his party alike are vulnerable.”
–Niccolò Machiavelli, The Prince (1513)
In case you missed it, a series of recent articles have made a Big Announcement:
The Semantic Web is not here yet.
Additionally, neither are flying cars, the cure for cancer, humans traveling to Mars or a bunch of other futuristic ideas that still have merit.
A problem with many of these articles is that they conflate the Vision of the Semantic Web with the practical technologies associated with the standards. While the Whole Enchilada has yet to emerge (and may never do so), the individual technologies are finding their way into ever more systems in a wide variety of industries. These are not all necessarily on the public Web, they are simply Webs of Data. There are plenty of examples of this happening and I won’t reiterate them here.
Instead, I want to highlight some other things that are going on in this discussion that are largely left out of these narrowly-focused, provocative articles.
First, the Semantic Web has a name attached to its vision and it has for quite some time. As such, it is easy to remember and it is easy to remember that it Hasn’t Gotten Here Yet. Every year or so, we have another round of articles that are more about cursing the darkness than lighting candles.
In that same timeframe, however, we’ve seen the ascent and burn out failure of Service-Oriented Architectures (SOA), Enterprise Service Buses (ESBs), various MVC frameworks, server side architectures, etc. Everyone likes to announce $20 million sales of an ESB to clients. No one generally reports on the $100 million write-downs on failed initiatives when they surface in annual reports a few years later. So we are left with a skewed perspective on the efficacy of these big “conventional” initiatives.
Adoption of a new way of seeing the world is one of the most difficult things to engender. Beyond simple intransigence, technical impedance, sunk cost thinking, fears and incentive mismatch, the dominant paradigms must be exhausted before people are ready to consider a new approach. We had to wait until the marketing tentacles of SOA, ESB and all of this other nonsense were struck down by lackluster results and expensive failures before the door was open to new ways of thinking. The ideas that have at least partially filled the gap include REST and Resource-oriented Architectures. Fortunately, these are gateway drugs to Semantic Web thinking. As Dan Connolly famously said (I think it was him), “The most important part of the Semantic Web remains the Web.” Organizations that aren’t prepared to embrace Web-oriented thinking are generally not going to be predisposed for Semantic Web thinking. Coming straight at them with ontologies is (and has been) a non-starter. So, as organizations are more effective at Web thinking, they will be more predisposed to Semantic Web thinking.
While we wait, we don’t hear about $20 million investments in Semantic Web technologies though. Is it because the technologies are unnecessary? Overly complex? Poorly understood? There may be elements of truth to these complaints, but at a more fundamental level, Semantic Web technologies don’t require consensus to use, and technologies that don’t require consensus to use don’t require $20 million budgets. If I have the capacity to accept your data as a spreadsheet, a relational database, a REST API, an XML document, etc. and extract content from it on my own, then I can do data integration on the Edge and I don’t need your permission to do so. To the extent that there have been Semantic Web successes, they have generally been behind-the-scenes successes. They do not require the kind of political and financial capital that these other huge initiatives do, and that makes all the difference in the world.
This makes it difficult to sell new products into this environment. When budgets are stretched and people fear change, it is difficult to justify software sales that don’t appear to comport with the status quo. There have been several companies who have attempted to build and sell Semantic Web tools into Enterprises who haven’t succeeded and some who have. I think the reasons why they do or don’t are far more nuanced than the authors of these articles usually consider.
The other problem is that the kinds of benefits that the standards enable aren’t exactly top of the list of many people within organizations. If we move our knowledge bases out of language-level domain models, relational schemas, frameworks, etc., we’ll end up writing less code. While any developer worth her salt would champion writing less code, many do not have that kind of foresight. They are not drawn toward technologies that seem to reduce their influence in the organization. Department heads whose groups produce information are equally suspect about the fluidity we are considering. If you free my data so that it can be used by people who don’t have to talk to me or ask my permission beforehand, then you are stealing my power as a cog in the system. I have seen this negative reaction in too many places to ignore that many organizations have employees actively working against larger goals and interests in favor of career-boosting tactics. This makes introducing new technologies into the organization even more difficult through no fault of the technologies.
Fundamentally, however, I think the problem comes down to the fact that the Semantic Web technology stack gets a lot of criticism for not hiding the fact that Reality is Hard. The kind of Big Enterprise software sales that get attention promise to hide the details, protect you from complexity, to sweep everything under the rug. And, as far as I can tell, these are basically lies told by people who want to lock you in. Trading off real complexity for the limitations of a tool that has its own shortcomings (artificial complexity) is not exactly a win. These lies are believed by people suffering from the delusional thinking of “We want to control, to own, to dictate but it’s too hard to actually think about everything all at once.” That centralization is an impedance to scaling and the flexibility to respond to constant change.
Even something as simple as a relational or XML schema is a simplification, a forgetting of complexity, a focus on only what is important in a certain circumstance. For specific uses, there is nothing wrong with this. In the big picture, however, this lossy storage and organization mechanism is a problem. To understand how organizations plan and make decisions requires an understanding of how Data becomes Information and then Knowledge and then Wisdom. We go from observing and recording to organizing, understanding and predicting. Knowledge and Wisdom are contextual, validated, prioritized orderings of Data and Information. If we ignore that context and treat knowledge management as simple information retrieval, then we are losing the value of the perspective of experts.
We come to conclusions, therefore, in contextual ways. Not everyone within an organization has the same needs of their underlying data. They may not have the same goals, abstractions or expectations. We currently attempt to solve the problem by extracting, transforming and loading (ETL) our data into separate systems to meet these different needs. This is a ham-fisted and antiquated way of thinking about it though. Semantic Web technologies allow a much finer level of granularity for reuse and expect information to be used in different contexts. Change is embraced at the most basic levels.
The desire to control manifests itself as a desire to own. We want to put everything into One Place so we can control it. Big Data helps enable these impulses, so it is a very popular set of technologies. No Big Data system is going to be Big Enough for Everything though. The contexts that used to be scoped to individual applications are now becoming less well defined. Targeting information for specific channels and users with different needs can dramatically alter what is important or valid. We need data models that support this fluidity. We aren’t going to get there by attempting to “master” our data into a single world view spanning all of the contexts.
The Semantic Web says: “Give up the pretense, you will never put All of the Things into One Place”. We don’t store the Web with our stuff. We store our stuff in the Web. It’s organic, flexible and, if you do it right, permanent enough that you can find things when and where you need them.
What we call things (entities, relationships, etc.) is driven by the language we use. An unexpected characteristic of these languages is that they do not serve a purely reflective role. Not every language of the world has the same word for every concept. This introduces the idea that language also has a constructive role in ordering our worlds. It would be great if we could come to consensus. However, even if we could do so within one organization (or one part of it), we are not going to do so across domain, organization, cultural, language and temporal boundaries. To pretend otherwise is to be doomed to repeat the history of failed IT initiatives.
These are all not technical problems per se and they are not going to be solved by technology alone. These problems are amplified by poor technology choices though (over and over and over again). Any technology that requires consensus is likely to fail on any number of levels. The Semantic Web technology stack allows us to agree to disagree without breaking existing systems. It allows us to get the efficiencies of consensus where we have it, but the freedom of independent thought and activity where we don’t. It enables Open World thinking, but allows you to close it as necessary. It affords us the ability to allow the same content to be used and reused in different contexts. If nothing else, Semantic Web technologies tackle these non-technical issues head on. They acknowledge that they will not solve your organizational dysfunction and deficiencies. They will allow you to make progress anyway.
Let’s assume for a moment that there haven’t been sufficient successes in this space yet to justify moving forward with these technologies (even though we know that there have been and continue to be). What is the alternative? If we abandon these ideas, what do we turn to? The answer after even the briefest consideration is that there is nothing else on the table. No other technology purports to attempt to solve the wide variety of problems that RDF, RDFS, OWL, SPARQL, RDFa, JSON-LD, etc. do.
People who do not know how or care to tackle large and scary prospects look to what successful companies such as Google and Facebook are doing. Neither of these behemoths are explicitly embracing the Semantic Web on record, but both are using and producing elements of it. They are both moving toward graphs and are looking to leverage standards-based ingestion of information. They also want to own the data and benefit from it without sharing it all back to you (which is the exact opposite of what the standards attempt to do). What’s good for these organizations is not necessarily what is good for you, but even they are down with structured metadata and distributed graph structures.
No one in the Semantic Web standards space thinks we have perfect solutions to these problems. But at least they are trying which is more than we can say for most of the rest of the industry. Adoption is low and slow, but it is also seemingly steady with new success stories emerging over time. Work needs to be done to improve the narrative, the tools, the visualization environments, etc. If the cost of Long Term and flexible thinking far outweighs the cost of doing (and redoing) it with a Short Term mindset, then these technologies will never become what they can be. In my day-to-day life of consulting across numerous industries, however, that simply does not seem to be the case. I’m finding teams “get it” when it is presented meaningfully to them. Adoption is not difficult, does not require $20 million budgets and demonstrates its value with ease.
Technologies that do not attempt to lock you in are rare. Technologies that embrace change and welcome dissenting perspectives are unheard of. Given that these are problems that every organization in the world faces and there is nothing else trying to address them, I’d say we keep on keeping on. Critics be damned.
“We who are young,
should now take a stand
Don’t run from the burdens of women and men
Continue to give, continue to live
For what you know is right”
–Curtis Mayfield, “Keep On Keeping On”Read more: Curtis Mayfield – Keep On Keeping On Lyrics | MetroLyrics
About the Author
Brian Sletten is a liberal arts-educated software engineer with a focus on forward-leaning technologies. His experience has spanned many industries including communication software, retail, banking, online games, defense, finance, hospitality and health care. He has a B.S. in Computer Science from the College of William and Mary and lives in Auburn, CA. He focuses on web architecture, resource-oriented computing, social networking, the Semantic Web, data science, 3D graphics, visualization, scalable systems, security consulting and other technologies of the late 20th and early 21st Centuries. He is also a rabid reader, devoted foodie and has excellent taste in music. If pressed, he might tell you about his International Pop Recording career.
