Hypermedia Types

When setting out to design a hypermedia type to solve a problem the designer needs to go through four primary decisions. These decisions set the tone for the hypermedia features available to clients and servers using the hypermedia type.

Base Format

What is the base format (base media type) of your hypermedia type. Most designers start with XML or JSON; usually not a good idea. It would be better to start w/ (X)HTML since that media type has hypermedia controls built in. XML and JSON have no built-in hypermedia controls and designers will need to create their own AND make sure all servers and clients understand these added features of the media type.

Application Domain

How will application-specific information be conveyed within the hypermedia type? How will clients and servers recognize domain objects ("customer-name"), links ("add-customers"), and other important details? Existing media types handle this differently.

1. NONE: HTML has no domain specificity built-in. None of the element names or attributes carry built-in application domain meaning (except possibly for "documents").
2. GENERAL: Atom has "general" domain specificty built-in. There are clear elements w/ meaningful names and selected elements have well-defined hypermedia meaning (rel="sef" and rel="edit", etc.). This general specificity works well for "generic" solutions. But sometimes Atom's limited domain semantics makes customizing the responses difficult.
3. SPECIFIC: VoiceXML has rather specific application-level elements. It is relatively easy to read a VoiceXML document and understand what the application does just from the element/attribute choices. Your solution may call for a very specific set of attributes and elements in order to express your app domain directly within the document.

State Transfer

How will clients send data back to servers? How will that transfer their own "state" values to the server for processing? Again, there are three basic ways to solve this problem:

1. NONE: SVG is a media type that supports no state transfer from client to server. It is a "read-only" hypermedia type.
2. PRE-DEFINED: Atom uses a "pre-defined" model for clients to transfer state to servers. The Atom documentation describes the format and elements that can be sent to the server. It is up to the client to "understand" these rules and have them "coded-in."
3. AD-HOC: HTML uses an "ad-hoc" model for establishing client-to-server state transfers. The HTML documentation defines a generic mechanism for sending arbitrary data to servers using the FORM, INPUT, and other related-elements. Clients must "understand" the rules for handling FORMs and the related controls as they appear within the response representations.

Process Flow

How do servers communicate valid process flow to clients? How do clients know what is possible as a "next step"? Once more, I'll suggest three possibilites:

1. NONE: HTML has no built-in process flow. It supports transfer protocol actions (GET and POST) and nothing else.
2. EMBEDDED: Atom uses an "embedded" model for process flow. The Atom Publishing spec details basic CRUD operations that are to be applied to existing elements in response representations (rel="self", rel="edit", etc.).
3. APPLIED: It is possible to define a set of process-flow definitions and "apply" them to an existing media type. For example, the ALPS model defines all the process-flow information for the XHTML media type using @rel and FORM descriptions that properly-coded clients can "recognize" in response representations.

Not every hypermedia solution needs every possible hypermedia option. These four basic decisions also influence each other. Selecting JSON as your base type means you start out with no state transfer, but likely have very domain-specific element and attribute design. Selecting XHTML means you get ad-hoc state transfer, but no domain specificity to start. Starting with XML gives you the power to design a very specific domain, transfer model, and process flow but you will have a very limited reach in terms of existing clients and servers that understand your hypermedia type.

Updated: 2010-10-07 @ 11:00

As an instructional/analytical aid, media types can be located on a continuum to identify their support for hypermedia behaviors. Unlike H-Factors which identify a set of elements that can appear within a media type, "H-Levels" focus on support for a set of desired interaction properties between client and server; properties that allow servers and clients to evolve independently without "breaking" or becoming unusable together. In reality, support for these properties means that servers can change responses to client requests in ways that allow for future modifiability without requiring clients to be updated, re-compiled, or re-deployed.

H-Levels are expressed from the viewpoint of the client application. IOW, "What happens to existing client applications when one or more of these H-Level items is changed?" The properties of a media-type that can foster support for this type of evolvability in client-to-server interactions are:

Content

The ability to add/remove elements in a message.

Address

The ability to change the URIs used by the client to make requests.

R/W Semantics

The ability to indicate the available read/write semantics including which elements are write-able, the protocol details to be used when requesting, filtering, and writing data, etc.

Appflow

The ability to modify the application flow (e.g. state-transition steps) using in-message control information including adding or removing state transition steps as needed based on the context of the current response.

It should be noted that not all the listed desireable evolvable properties (Content, Address, R/W Semantics, Appflow) are requried to build successful client-server interactions. Also, the resulting ranking is not designed to indicate quality (the higher ranking, the better), but simply help identify target media types along a concrete continuum.

H-Levels : A Table Summary (Read from bottom to top)

H-Levels : L0 - L3

Below is a list of four levels along with common characteristcs and thier affect on the way clients must be coded and how these client must interact with servers.

L0 : Serialized Objects

Example media-types: XML-RPC, SOAP

Each interaction between client and server results in a unqiue message body. The details of which message body is to be sent for each URI request and which message body is returned in response to that request must be outlined in the documentation and then encoded into the client. Some media types (SOAP) support a consistent "wrapper" around the varying responses. The URIs, request and response bodies, the read/write semantics, and the application flow are all documented before hand and then expressed in the source code of the client.

Changes to the following items usually require changes in client code:

• Content
• Address
• R/W Semantics
• Appflow

L1 : Standardized Data Format (SDF)

Example media-type: CSV, XML, JSON

Clients and servers share a common message format for all requests and responses. This means each request and response body can be validated for correctness against a documented format. The format is designed to allow for adding/removing elements safely without breaking existing clients or servers. While the data format is consistent, the URIs used, the read/write semantics, and the application flow are all documented before hand and then expressed in the source code of the client.

Changes to the following items usually DO NOT require changes in client code:

• Content

Changes to the following items usually require changes in client code:

• Address
• R/W Semantics
• Appflow

L2 : SDF + Links

Example media-type: Atom + AtomPub, OData

Along with the shared data format for requests and responses, servers include URIs within the message bodies themselves. Clients use the URIs supplied by the server to accomplish state-transitions. In this way, servers can safely change the URIs without requiring re-coding the client applications. In this approach, the read/write semantics and application flow are all documented before hand and then expressed in the source code of the client.

Changes to the following items usually DO NOT require changes in client code:

• Content
• Address

Changes to the following items usually require changes in client code:

• R/W Semantics
• Appflow

L3 : SDF + Links + Application Controls

Example media-types: HTML, VoiceXML

In this model, each response message sent by the server includes application control elements (including URIs) that instruct clients on the details of read/write semantics and possible application state transitions available at the present moment. Clients can locate and use these application controls in order to (when appropriate) display options to users or automatically select the proper transition based on a pore-determined goal for that client application (e.g. bots, etc.).

The documentation for this model outlines the agreed data format and details the possible application control elements that can appear within a response and the possible read/write semantics associated with each application control. Clients are coded to locate and understand the application controls and act accordingly.

Changes to the following items usually DO NOT require changes in client code:

• Content
• Address
• R/W Semantics
• Appflow

WARNING: There be dragons ahead. Don't go here unless you are sure you'd rather 'roll your own' [hyper]media type instead of using an existing, properly registered one such as [X]HTML or Atom.

Creating a custom media-type for internal use does not need to be very complicated. If you are working on a localized application and want to get everyone using the same state-transfer contract, you can use a simple media type that has just a few 'hooks' and allows for future expansion as your needs grow.

Object Container

An easy start is to adopt a simple data format that takes on properties of both [X]HTML and Atom. The <body /> portion can hold a serialization of a custom object (ala Atom-style) at the beginning. This makes it easy for developers to use existing frameworks when crafting messages. Here's a an example 'skeleton':

<root>
  <meta>
    <!-- meta-data, system info, etc. can go here -->
  </meta>
  <body>
    <{custom-object-goes-here} />
  </body>
  <error>
    <!-- place any error details (text, stack trace, etc.) here -->
  </error>
</root>
          

Standard Data Representation

The next step is to standardize the representation of data in the <body /> section of the message. IOW, eliminate the use of the object serialization pattern and use an agreed data format that is not bound to server-side storage|object-types. One simple format is to adopt a generic flat name-value pair (NVP) pattern:

<root>
  <meta>
    <!-- meta-data, system info, etc. can go here -->
  </meta>
  <body>
    <data name="email"></data>
    <data name="username"></data>
    <data name="date-created"></data>
  </body>
  <error>
    <!-- place any error details (text, stack trace, etc.) here -->
  </error>
</root>
          

Using a dedicated data format like this will add some work for developers; they will need to drop the habit of blindly serializing internal objects as public messages. However, once a simple conversion utility is built, it is relatively easy to read/write a data format like this. Even better, if you need to add new elements, they can appear as one more in the collection. This makes evolving the messages trivial.

Hypermedia Controls

In the previous examples, all hypermedia information must be encoded in the client and server themselves. For example, knowing what data to write, which URI to use when writing, which HTTP method to use when writing, etc. However, you can greatly increase the value of your data format by adding this semantic information into the messages themselves. This takes more planning and requires additional coding (mostly for clients), but can pay off in the long run. Here's a simple example:

<root>
  <meta>
    <!-- meta data, system info, etc. can go here -->
  </meta>
  <body>
    <!-- allow searching for one or more users using HTTP GET -->
    <query href="..." rel="search">
      <data name="username" />
    </query>
    <!-- allow sending data to the server, if rel contains "update", use HTTP PUT -->
    <write href="..." rel="user update">
      <data name="email"></data>
      <data name="username"></data>
      <data name="date-created"></data>
    </write>
    <!-- if rel contains "add" use HTTP POST -->
    <write href="..." rel="user add">
      <data name="email"></data>
      <data name="username"></data>
      <data name="date-created"></data>
    </write>
  </body>
  <error>
    <!-- place any error details (text, stack trace, etc.) here -->
  </error>
</root>
          

Using Hypermedia controls in the message means clients need to be coded to search for and understand the various pre-defined controls (in this example they are "query" and "write"). Although this takes additional work at first, it means future messages can be crafted w/ new or changed semantics w/o requiring a rewrite of the client code.

Going Beyond Simple

This is example Hypermedia format, while useful, is very simple. There are lots of features that could be added, may be needed to make this a useful media type. The point is that, starting this way means you can create a focus on using media-types as a fundamental aspect of your overall architecture. By doing so, you get a whole bunch of 'win' without that much initial cost.

I have seen several examples where folks give media type examples that look like this: application/customers+xml, etc. At first, this seems like a good idea, but in the long run, it is not. media-types should not be based on "objects" that might exist in the server-side model.

Instead, media-types should be designed at the application level; possibly the "task" level. IOW, application/accounting+xml or application/shopping+xml, etc.

Here are some tips to keep in mind when designing a hypermedia type.

Don't use actual URIs

When modeling your hypermedia type, esp. when creating examples, DO NOT include actual URIs in the samples. Instead, use a placeholder token (e.g. {uri}).

Favor Generic Elements

Generic element names w/ specific attributes are easier to process for most apps and they offer more extensibility to application designers. Instead of specific element names (<name /> <email />, etc.) use a generic element along with a name attribute (e.g. <data name="name" /> <data name="email" />, etc.)

Plan for Extendability

Design your media type to allow developers some freedom to extend it in ways that you might not think of. For example, don't create such a tight set of schema rules that application developers are unable to add new elements or inputs as needed.

Avoid Single-Protocol Designs

Using a keyword or element (e. g. POST) that is tied too closely to a single protocol (e.g. HTTP) may limit the application of your media type in the future. Instead, use generic keywords (WRITE, etc.) or elements that can be "mapped" or "bound" to an application protocol in the implementation phase.

Information should have these four aspects. The more aspects, the higher the value of the information message itself.

1. Structured
   Messages need structure to provide context. With context, agents (humans, machines, etc.) can derive meaning, establish provenance, and assess trust. Structure can be expressed as control data (e.g. HTTP headers) and/or meta-information within the message itself (e.g. markup).
2. Linked
   Messages should contian links to other information. Links are another way to add context to the message. Links should be more than arcs from one piece of data to another (e.g. HTNL A tags). Links should also express other relationships such as "fetch and display here" (e.g. HTML IMG tags), "make a request with the following additional inputs" (e.g. HTML FORM w/ GET), and "write the following data to the server" (e.g. HTML FORM w/ POST/PUT/DELETE). Links inform agents of what they can do, not just what they can see.
3. Portable
   Messages should be portable. They should move freely between locations. Each message should contain all the information needed for agents to use to render and process as needed. Messages should be able to adapt to the device and context at hand (e.g. content negotiation). Messages should be "re-playable" meaning that they can be stored and accessed at a later date and still successfully render as expected.
4. Autonomous
   Messages should be autonomous. They should not be overly dependent on additional messages or their origin server, etc. Messages should not be dependent on separate application programming in order to be useful. IOW, the message is the application. Network-compatible agents should be able to find, consume, and operate on the message - no matter where it comes from. Messages should not be dependent on custom code that must first be installed on the agent.

Employing hypermedia types on the WWW has a number of advantages:

Address Abstraction

Hypermedia types use designated elements (A, FORM, IMG, etc.) and link relations (@rel) as 'tokens' for clients to watch for. The actual contents of the associated URI is not important to the client. This means the URIs can safely change over time.

Protocol Abstraction

Hypermedia types can be designed as protocol agnostic. Links can use any protocol supported by the client and can safely change over time as needed.

Object Model Abstraction

Hypermedia types can expose state representation transfers that need not match to any object model or server-side storage model. Passing representations means the server can safely change it's internal object/storage details without breaking the client.

Workflow Abstraction

Hypermedia programming allows sharing representations and links. That means servers can, over time, safely change the flow of an application by introducing links to new state transfers or changing the order of the state transfers.

Function Call Abstraction

Using state transfers means data passed between clients and servers can be in different formats, based on the use case requirements. That includes typical name/value pairs (name=mike&size=large...), binary objects (e.g. image transfers), or even direct transmission of data blocks (e.g. XML, JSON, etc.)

Asynchronous Evolvability

Using Hypermedia types means authors can add new features to the messages without breaking existing clients. The media-type itself can be safely "versioned" in ways that will not break deployed clients, too. This means that new clients can take advantage of the new features while co-existing with the old clients.

Client-Driven Composability

Clients that understand a wide range of media types can "compose" their own work by leveraging multiple' servers, negotiating media types and, in effect, creating an individualized "program" for the client to follow. Hypermedia types encourage this "composability" since servers can offer up data via well-designed hypermedia types and allow clients to determine what tasks are done at which locations using the hypermedia control information provided in server responses.

http://www.example.org/q1w2e3r4t5

<img src="http://www.example.org/q12w3e4r4t5" />

<a href="http://www.example.org/q12w3e4r4t5">Home</a>

<form action="http:/www.example.org/q1w2e3r4t5">
  <input name="search" />
  <input type="submit" />
</form>

Each [hyper]media-type allows the author to add meaning to URIs in appropriate ways. The more rich the expression of URIs, the more powerful the [hyper]media-type. [Hyper]media-types that have only a few ways to provide meaning to URIs have limited value on the Web. Those that provide no structure (meaning) to URIs are not [hyper]media-types at all.

As a [hyper]media-type's native functionality (it's built-in semantics for data, read-write, and presentation) is percieved to fall short of user|developer expectations, code-on-demand (COD) will be employed in greater frequency and size . IOW, if the native semantics fall short, devs will use custom coding to compensate.

It's important to note that it is the perceived shortcomings of the media-type that drive this increased reliance on COD. Poorly-trained|informed developers may needlessly drive up the COD value of a response. However, legitimate increases of COD usually indicate the media-type employed is inappropriate for the needs of the user|developer. In this case, either a different, existing media-type that has the proper semantic values (re: data, read-write, presentation) should be used.

If|When the community is in general agreement that there is no other suitable media-type for the task, it is then time to consider evolving the media-type to fit the needs. The\ W3C has drafted guidance on how to accomplish this goal.