mca blog

i think i see this question (in some form or another) almost once a week. in forums, online chats, blogs, in real-life, etc.

"For anyone using JSON-based media types, are you passing around a '__type' property in order to deserialize? If not, how do you parse the JSON (i'm working in .net here)"

first, this is not a silly question. in fact, the second sentence goes to the heart of it:

"If [you don't use a 'type' property], how do you parse the [payload]?"

and here is one of the real gotcha's of distributed network programming: type-system problems. there are several common ways to try to solve this seemingly unsolvable problem. let's look at some of the persistent attempts:

"typing" via schema

the best-understood way to solve this problem is to publish a detailed schema document that lists arguments and interaction details.

<?xml version="1.0"?>
<definitions name="StockQuote"

targetNamespace="http://example.com/stockquote.wsdl"
          xmlns:tns="http://example.com/stockquote.wsdl"
          xmlns:xsd1="http://example.com/stockquote.xsd"
          xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/"
          xmlns="http://schemas.xmlsoap.org/wsdl/">

    <types>
       <schema targetNamespace="http://example.com/stockquote.xsd"
              xmlns="http://www.w3.org/2000/10/XMLSchema">
           <element name="TradePriceRequest">
              <complexType>
                  <all>
                      <element name="tickerSymbol" type="string"/>
                  </all>
              </complexType>
           </element>
.....

this is the way SOAP was designed to work, but shared schema is used in other environments, too. the advantage of this approach is that it provides clear description for all parties.

the downsides is that it most often is used as a way to express the details of private objects (almost always on the server). when these objects change, all previously deployed clients can become broken and must be rebuilt using the newly-published schema document. i've heard people say this "compile-time" binding is not a requirement of the SOAP model, but i've yet to find an example of this happening in a dynamic way that is scalable.

"typing" via URI

another common solution is to try to make the URI carry "type" information:

http://www.example.org/orders/123

this is a popular solution since most frameworks can easily generate these URIs from metadata within the source code. it also makes things relatively easy for frameworks that use templating and other processing based on the "type."

of course the biggest bummer is that you loose control of your public URI space. now private source code decides what your URIs look like and any change to source code threatens to invalidate previously published (and possibly cached) URIs.

"typing" via payload

if you want to keep in control of your URIs, you can opt for the second-most common option i see: decorating the payload. this is usually done by adding a "type" element to the JSON or XML body or, in rare cases, adding a <meta> tag or @profile attribute to (X)HTML documents.

	{"__type":"Circle:#MyApp.Shapes","x":50,"y":70,"radius":10}

i see this quite a bit in cases where JSON is used at the data format for payloads. it has the distinct advantage of allowing frameworks to "sniff" the contents of the payload to find the "type hint" and process the data accordingly.

the primary drawback of this method is that the payload is now tightly bound to source code. and, again, changes to the source code on the server will modify response payloads and invalidate incoming request payloads from clients who have no idea the server has changed the details of the "typed" object.

"typing" via media type

the other common solution i see is to create a custom public media type for each private object on the server.

....
...
@PUT
@Consumes("application/stockquote+xml")
public void createStock(Stock stock ) {
......
}
........
@Provider
@Produces("application/stockquote+xml")
public class StockProvider implements MessageBodyWriter<Stock> {
.....

the advantage here is that you have full control over both the public URI and the payload (although most frameworks link a private class to these custom media types).

the downfall of this approach is that it merely pushes the problem along; in this scenario a single application could generate dozens of new media types. it would be an incredible feat for an independent client application to try to keep up with recognizing and parsing the thousands of media types that might be produced in a single year.

but the real problem is...

by now you should see a pattern. all these attempts are focused on the same thing:

"How can a server successfully export it's private objects in a way that clients can see and use them?"

you might think that this is the correct question to ask, but you'd be mistaken. a better question is:

"How can a server and client share a common understanding of the payloads passed between them?"

and the first answer is to stop trying to export private information out onto the public network.

so what's the answer?

instead of being stuck on the notion of shared objects, distributed network clients and servers need to focus on shared understanding; a vocabulary or ontology.

currently, for HTTP-based applications the best way to provide a shared understanding is through well-defined media types. and a media type SHOULD NOT be bound to any private objects; it SHOULD NOT rely on any pre-determined URI patterns. instead, the media type should encapsulate the domain for which the client and server are communicating; it should be able to express the appropriate application-level semantics needed to accomplish the domain's tasks and describe the results.

i, personally, prefer using and designing Hypermedia Types. they offer the possibility of expressing not only the domain-specific details of shared understanding, but also the protocol-level elements (through a set of H-Factors.)

oh yeah, Maslow's hammer...

i bet you wondered when i'd get around to the Maslow thing...

while the notion of using hypermedia-rich media types that also express application-domain details is appealing to many, the road to this apparent nivrana is not yet well-paved nor well-travelled. most frameworks are tuned to serializing private objects out to the public network using one or more of the methods i mentioned here. i know of very few frameworks that have avoided the 'object pit' (WebMachine being one of them).

so, it's understandable that developers are likely to ask how to handle private object serialization and sharing; they simply haven't found many alternatives. developers can become quite attached to their 'tools' (editors, frameworks, programming languages, etc.) and, as Maslow observed in 1966, in cases where those tools are not quite adequate for the job at hand, it is tough to drop your hammer and reach for another, possibly unfamiliar, instrument.

but that is exactly what i am urging you to do when you work on distributed network implementations like HTTP over the Web.

stop.

step back.

drop the hammer of object serialization.

and try picking up the tool of hypermedia types instead.

it just might be more suited to the job.