How Do We Get to the Semantic Web -- Radar Networks or Blue Organizer?

No! In spite of what several companies will have you believe, we are not there yet. We are taking small steps toward the so-called semantic web, which is to have Web services and sites gain understanding of human and machine interactions, preferences and patterns.

I presented a few examples of what's going on in a recent piece for The Economist: Some Antics for the Web. To be sure, there are many other actors out there besides IWantSandy, TripIt and Wesabe. They approach the challenge by either brute computing power, or by finding smart work-arounds that rely on an 80/20 solution. Humans still do a lot of the tagging, grouping and making sense of data. It will stay that way for quite a while.

What we are looking at, then, promises to be a semi-autonomous network where software can be addressed in conversations and asides, almost as humans would, and where automated agents draw inferences and connections between the electronic breadcrumbs their users leave.

The programmers and engineers at the forefront of this new web argue it will far exceed the functionality of today’s Web 2.0, the crop of interactive applications. The semantic web will possess reasoning capabilities, enabling it to wring structure and thereby meaning out of reams of unstructured data – from in-boxes to travel plans and bank statements.

Sir Tim Berners-Lee described this vision back in 2001 in a seminal article for Scientific American, in which he spoke of software agents interacting with humans and among themselves. The technology and standards to make this goal come true have been evolving ever since. First, however, there has to be data and metadata – or descriptions of that data – that new software can parse, categorize, and connect.

How much machine intelligence is necessary to give oceans of data meaning? Building ever more powerful algorithms to achieve true natural language processing and search is a lofty goal, as Wesabe's Marc Hedlund told me. Yet it's not always necessary.

Users are extremely good at sorting data for selfish reasons, and the emerging glimmer of meaning helps all other users, humans as well as computers, make better sense of it. Hedlund compares it to an exoskeleton where the human wearer lifts a finger and the machine amplifies it into a powerful gesture.

There are companies taking the high road to the semantic web, among them the new buzz-magnet Powerset. The SF-based company is readying an ambitious expert system for Web searches that's based on natural language technology developed at Xerox PARC. The service won't go live until 2008, but it has already captured the imagination of techies who are dreaming of the ultimate search engine that presents answers, not just lists of possible results like Google.

It might never happen, Alex Iskold told me over the course of several conversations. He is a prolific tech blogger over at Read/Write Web and CEO of Adaptive Blue in New York. He thinks the focus on brute processing power to imbue the web with more and deeper meaning is a misguided waste of resources. Instead, his company is also following the baby step approach. It offers one of the most popular extensions to the Firefox browser, which observes a surfer’s moves and offers up context for almost every word on a web page.

In order to do that, the plug-in can understand 30 concepts from books to stocks and related activities, what Iskold calls the “nouns and verbs” of the web. By connecting a running tally of someone’s preferences with his browsing habits, the software can infer recommendations. Over time, Iskold thinks his service offers a low-key road to the semantic web by just using simple heuristic models.

No matter whether it can be done by machine intelligence alone or not, the age of intelligent personal agents who don’t rely on training wheels is at least a decade off, according to serial entrepreneur Nova Spivack. He gave me a demo of his much-awaited semantic web service called Radar Networks that's slated for an October launch.

Radar wants to turn a person’s informational flotsam into one smart wiki that automatically links people, places, pictures and plans into a futuristic information hub. Think of it as cyber kefir with a brain-- it just keeps growing on its own.

Since a picture is worth a thousand words: Spivack has a great chart of how to get to Web3.0 and the semantic web on his blog, which I have reproduced on the left (Source: Radar Networks & Nova Spivack, 2007).

Comments

[Alex Iskold]

Great and thoughtful summary - as always!

I will be blogging about these topics next week on Read/WriteWeb.

[Avi]

Very informative article about the future of the semantic web and the different approaches towards its development. You might be interested to hear about a natural language technology that will be used to create lexical semantic databases, as part of the process in developing the semantic web. The technology is called Streaming Logic and it has been developed by an Israeli company called Linguistic Agents, that I do media relations for. They have a very sophisticated parser, that parses based on the theoretical linguistic principles of Chomsky, that they will be using to build lexical semantic databases. You can visit their website for more information if you are interested - www.linguisticagents.com. I look forward to reading more of your insights in the future on the development of the semantic web.