communicating with aliens and the Licklider Turk

2013-08-18 @ 22:11#

there are times when, as i am talking to some people about about Hypermedia and REST, i get the feeling i am talking to aliens. this happens when it becomes apparent that what i mean when i talk about these things is not the same thing others think i mean. it used to unnerve me. now, i am mostly intrigued and usually entertained.

asking questions

when it becomes clear that we're not sharing the same basic understanding of things (e.g. REST !== HTTTP, Hypermedia is not limited to HTML, etc.) i usually stop, take a moment and then start asking probing questions. things like:

  • Do you think HTTP and REST are the same thing?
  • Do you think REST and CRUD are the same thing?
  • Do you think HTTP and CRUD are the same thing?
  • Do you think REST is a standard; that there's an RFC for it?
  • Do you think links and forms can only appear in HTML responses?
  • Do you think APIs cannot include Hypermedia controls in responses?

there are many other Qs, but these usually have the effect of getting folks to provide responses that help each of us focus in on a shared set of terms and assumptions that make a discussion enjoyable instead of a wrestling match.

Licklider's "problem"

and that process came to mind as i listened to some of Bret Victor's excellent talk, The Future of Programming. about mid-way through the talk, he references J. C. R Licklider's 1963 paper "Memorandum For Members and Affiliates of the Intergalactic Computer Network" where he speculates on how a giant network of computers will need to handle the case where two machines have "just met" and need to work out how to communicate successfully with each other.

Here's an excerpt of what has become know as "The Communicating with Aliens Problem":

Consider the situation in which several different centers are netted together, each center being highly individualistic and having its own special language and its own special way of doing things. Is it not desirable, or even necessary for all the centers to agree upon some language or, at least, upon some conventions for asking such questions as "What language do you speak?" At this extreme, the problem is essentially the one discussed by science fiction writers: "how do you get communications started among totally uncorrelated 'sapient' beings?"

J C. R. Licklider (1963)

basically, the two machines need to "probe" each other, sharing data back and forth and working to identify "shared understanding" that can be the basis of further communication. there are many implications and complications in this simple idea but the essence is sound: if you want to make sure to communicate successfully, you need to run some tests and establish some shared understanding.

doing things the hard way

the same problem exists for any machine-to-machine communications. there needs to be some a priori understanding, some shared information, before two machines can start. this is the way things work today. machines need to share understanding ahead of time.

the most common way to share this understanding is for machines called "servers" to expose an interface (API) and then have some human write lots of documentation that other humans can read in order to program other machines called "clients" to use the interface to get something done. Bret Victor points out in his talk that this process (human codes server, writes documentation, other humans read docs and write clients) doesn't scale. and i think he's correct.

what is needed is a way to reduce the human dependence; come up w/ ways to get the same kind of results w/o the same effort from humans. and i think that is also quite possible.

we have language models

lucky for us there is at least one network protocol that supports a number of "probe" options in ways that could help solve Licklider's problem: HTTP. this protocol has the notion of content negotiation built in. unfortunately, most HTTP devs miss the point of this feature and/or ignore it completely. by establishing a series "languages" that machines can use to speak to each other, HTTP can be used to allow machines to "probe" each other and discover which "languages" they share.

again, this "language" model already exists. HTML is one of the more powerful examples. it has the ability to do more than communicate data. it can also communicate possible actions via the use of Hypermedia affordances (e.g. links and forms). and it is not the only "language" that we have today to handle this task. some of the more recently registered languages include:

and there are other, as yet unregistered, languages machines can use to communicate not just data, but also actions. not just the "nouns" but also the "verbs" with which to share information between machines.

we have vocabularies

of course, the use of a protocol that allows for negotiating a mutual language model is just part of the solution. another important feature is the ability to share a vocabulary between parties. to use humans as an example, even if we both speak French, we may still be unable to share understanding if you are talking about medical topics and i have no idea of the subject whatsoever. to solve this, we need shared vocabularies, too.

and, of course, we have them, too. here are a few of the most commonly used vocabularies:

and RDF offers a rich model for defining and expressing new vocabularies. RDF is a meta-model for vocabularies.

the Licklider Turk

so, we have a protocol that supports negotiation, we have a set of languages machines can use to communicate, and we have a set of vocabularies that machines can use to share information using those languages over a protocol. well, have we solved the problem?

hardly.

in fact, we continue to assume a kind of "mechanical turk" form of the internet where there is always a human (actually many humans) hidden in the underworks making the machines appear to be communicating. it seems like an elaborate parlor trick.

as long as we continue to rely on hidden humans to make the Web work we'll be limited by the scope and power of any single human. instead, we need to be thinking beyond the human limits and start working to reduce the dependence on humans to define and animate server interfaces (APIs) and stop the practice of using humans to create one-off custom clients that only work w/ one of the server interfaces created by some other human.

ALPS - a possible solution

over the last year, i've been puzzling over this problem. why haven't we gotten further along in building systems where machines have more freedom and more ability to negotiate interfaces and select shared vocabularies? i think the reason is that the three pillars of this process (protocol, language, vocabulary) have not yet spent enough time working together to create workable solutions. i've seen great efforts within some of these camps (REST in the protocol space, various media types in the language space, RDF in the vocabulary space, etc.). but i've not seen much evidence of successful inter-disciplinary work. maybe this is going on and i'm missing it. if so, i am looking for pointers.

in the meantime, i and Leonard Richardson have been working on a model called Application-Level Profile Semantics or ALPS. this is an attempt to capture both vocabulary and action information in a protocol- and language-agnostic manner. this allows machines to express the same information "profile" over many media types (languages) and protocols (HTTP, CoAP, WS, etc.) without losing any fidelity within the vocabulary.

this is an early effort and i am happy to hear from folks who have comments and suggestions in this space. i'm especially interested in anyone who would like to join us in working on the ALPS project. we already have a few people working with it and the feedback has been very helpful.

preparing for the Intergalactic Computer Network

there you go, from "Communicating with Aliens", through the Licklider Turk, on up to the ALPS effort to address the problem posed by the existence of an Intergalactic Computer Network. there are lots of opportunities here!

ALPS