mca blog

The H Factor for Media Types

Tue, 09 Mar 2010 20:24:00 GMT

today i stumbled upon The Hypermedia Scale posted to REST Patterns by @bjorg last year. it reminded me of some work i've been doing recently on cataloging Hypermedia Types.

unlike the REST Patterns article, which categorizes hypermedia based on CRUD capabilities, i've tried to parse out various features of a media type that allow servers and clients to share information about hypermedia capabilities. i've dubbed my effort The H Factor.

currently, i have identified two classes of factors: link support, and control data support. here's my current breakdown:

Link Support

Embedding links (LE)
e.g. HTML img tags
Outbound links (LO)
e.g. HTML anchor tags
Templated queries (LT)
e.g. HTML form w/ method='get'
Non-Idempotent Updates (LN)
e.g. HTML form w/ method='post'
Idempotent Updates (LI)
e.g. XmlHttpRequest object w/ PUT or DELETE

Control Data Support

Control data for reads (CR)
e.g. XInclude's accept attribute
Control data for updates (CU)
e.g. HTML form w/ enc-type attribute
Control data for methods (CM)
e.g. HTML form w/ method attribute
Control data for link relations (CL)
e.g. Atom link elements supports rel="edit" and others

by identifying elements of a media type that support link semantics and detailing the ways clients and servers can decorate these elements with added information to modify the method used and the control data supplied, media types become more than representations of data, they become the instrument of application flow for both the client and server.

that is the essence of a hypermedia type.

the developer certification problem

Sun, 07 Mar 2010 15:42:00 GMT

i saw a great post from @unclebobmartin on dealing with the terrible state of software developer skills. it reminds be of an effort i got involved in several years ago to promote government licensing of software developers (ala Steve McConnell's "After the Gold Rush"). that fell flat, of course.

IMO, the issue of poor quality software is tied up in how developers want to 'cast' their work. IOW, is the discipline of writing code primarily art, craft, or engineering?

if software is an engineering discipline, then the way to ensure quality and value is through boards and standardized testing. the way architecture, structural engineering, and other disciplines are managed.

if software is a craft, then the way to ensure quality and value is through a stringent master/apprentice model the way many building trades (plumber, electrician, etc.) are handled.

if, on the other hand, software is primarily an art, then it's future lies in the establishment and promotion of 'schools' and styles and standard-bearers similar to the way most other artistic endeavors are judged and sustained.

personally, i favor the 'craft' model. in this case, what is currently needed are master developers willing to take on apprentices. these masters need to be of the caliber that industry accepts their advice and favors their apprentices. in this way, masters can protect and promote the quality of software development for future generations.

IMO, Robert Martin would make a fine master developer. i'm sure he can identify others worthy of that title. from there, it's up to those masters to take on the responsibility of training apprentices and making sure they carry forward the high craft of software development.

of course, we need folks willing to accept the role of apprentice, too. people willing to figuratively 'sit at the feet' of the master and take in all the master has to offer. from what i've seen over the last several years, there are too few developers willing to accept that relationship. instead they are anxious to create 'the next big thing' in software. the result is usually a weak idea poorly implemented. occasionally i see a good idea poorly executed. rarely do i see evidence of a great idea implemented by true software craftsmen.

Fielding Property Maps

Wed, 03 Mar 2010 18:06:00 GMT

I posted some example code along w/ a short explanation today. It demonstrates an HTTP resource update pattern based on an idea by Roy Fielding. At the end of a post on the topic of WebDAV semantics, Fielding illustrates one way to expose and manipulate a set of properties for a resource using a URI convention (similar to that used for HTML image maps) where appending ;props to a resource URI allows clients to access any available properties for that resource. Since I first saw this idea articulated by Fielding, I've dubbed the pattern Fielding Property Maps (without his knowledge or consent).

the interactive example can be used to modify the metadata for the poem document using the Fielding Property Maps pattern and supports the following operations from any HTTP-aware client:

GET     /poem;props           retrieves a list of properties for that document
GET     /poem;props?{dcterm}  retrieves a single property for that document
PUT     /poem;props?{dcterm}  supports updating the value the indicated property
DELETE  /poem;props?{dcterm}  deletes the value for the indicated {dcterm}, 
                              but does not remove the actual metadata entry.

thanks to Jan Algermissen for showing me Roy Fielding's email post on the subject.

"PATCH is a very simple method"

Mon, 01 Mar 2010 12:28:00 GMT

Regarding PATCH Method for HTTP (RFC5789):

PATCH is a very simple method -- look at the content-type to see the format of the body, which contains instructions on how to apply a desired state change. The intent (when I added it back in 1995) was to define patch media types that contained a first section containing changes to metadata and then later sections containing the diffs. The decision on which patch media type to support would largely depend on the nature of the resource (and the ability of folks who wanted them to get off their duff and define the types). I started on a definition of a patch format for YAML late last year, but it was derailed by the partitioning work.

....Roy (March 2008)

You have a RESTful application

Tue, 23 Feb 2010 11:01:00 GMT

If you can draw a state machine in which:

Each state is self-described (resident on the client)
The transitions from that state are also self-described (instructions for the client to initiate)
And each transition is invoked using a self-descriptive message

Then you have a RESTful application.

....Roy from rest-discuss, late 2004]

thanks to @algermissen for the pointer.

"I Design Representations"

Mon, 22 Feb 2010 10:52:00 GMT

"And now for something completely different..."

Monty Python

It was late afternoon on a Friday and, against my better judgement, I'd agreed to meet some friends at their favorite geek watering hole. Even at this early hour, the place was packed with twenty-something cube-rats eager to get a head start on the weekend. I had the misfortune of being the first of my clan to arrive so I waited quietly at the bar; nursed my beer, ignored the pudgy guy next to me, and kept an eye on the entry-way for a recognizable face.

"So, what do you do?" pudgy finally asked me as he slurped his whiskey on the rocks.

I gave him a quick look. Balding, cheap sport jacket over khakis, rumpled shirt, and an ugly tie. "I design representations for a living," I responded bluntly.

"A designer, eh? 'Fashion Forward' and all that?" he asked waving his drink in my direction.

"Representations," I repeated, leaning in to be heard over the thin gruel of 80s rock pouring from the docked iPod behind the bar. "I work with representations." The smell of burnt pizza lifted from his sport coat.

"Oh, I get it," he nodded. "My brother in-law is a lawyer. Represents sports celebrities. Lots of negotiations and stuff like that."

"Well, I work with content negotiation, not contracts."

'Pizza 'n Whiskey' shot me a puzzled look. "No law degree?"

"Nope. Computer Science."

"Ah! Computers!" he intoned as he took a gulp of brown liquid. "All about outsourcing nowadays, eh?"

"I do Resources, not outsourcing."

"Yeah, we're in the same game," he continued, ignoring my last remark. "I'm a Program Manager on that big integration project up at the university." Proudly stabbing the air with his free hand, he told everyone within earshot, "My team 'owns' the middleware!"

"Really?"

"Oh, yeah. Over ten months to spec it out; coming in at two years for the first release. Huge project." He stretched out the word 'huge' and ice clattered in his glass as he carefully traced a giant circle in the air between us.

"Sounds like exciting work."

"Never-ending work is more like it," he replied, turning to me and squaring his shoulders. "See, we barely get one part of the system done and then the users tell us they need some other piece of data on the screen or report or something. That means I gotta tell the programmers in India to start all over again." He counts out on his fingers. "New data tables. Rebuild the object layer. Update the business rules. Change the UI. Compile it. Test it. Deploy it. That stuff takes time, right? Hell, before it's all done the customer wants something else!" He paused, hitched his thumb under his belt and added, "Of course, we bill by the hour, so..."

"And lucrative." I was egging him on, now.

"Say, you're a computer guy!" His eyes lit up. "You know anything about ORMs? We could use another ORM guy. The last one quit on us."

"No, sorry. I'm an HTTP guy."

"Oh, well hey! We got HTTP stuff, too. All 'Ajax' this and 'Two-Dot-Oh' that." He used 'air quotes' to punctuate the buzz words for me. Then, as if taking me into his confidence, he reached up and put his arm around my shoulder offering me another hint of pizza and said quietly, "Frankly, I think that's all just fluff but the customers love it."

"Fluff?"

"Heheh, that's OK," he backed up and tossed me a big smile. "Every time they ask for that 'webby' stuff we end up bulding a whole 'nuther set of middleware." He waved his hands in the air, saying "One for the desktop, one for the browser." He looked like Moses parting the Red Sea. "It's great for the monthly billings, but a real mess on the back end." His voice trailed off as he contemplated the dwindling whiskey at the bottom of his glass. "Yeah, sometimes this web stuff can be a real pain..."

I felt a bit sorry for him. "It doesn't have to be that way, you know."

"I know. I know," he confessed. "I keep sayin' we need some way to write just one set of middleware that everyone can use. But every time we try it the Web guys complain, the desktop guys are unhappy, no-one can agree on a single data format, our perfomance metrics go to hell... Plus, every little change means we have to re-write all the client code, do a crap ton of tests again..." He slumped against the bar and stared off into space. "If only there was a way to do middleware differently. One that scaled well, didn't take forever, and let everyone get data the way they wanted."

I smiled.

After a short pause, his eyes slowly narrowed as he pointed his glass at me and asked, "What is it you do again?"

"HTTP. Resources and Represenations, remember?"

"Huh... Damned if I even know what that means." Shrugging his shoulders, he thrust his right arm out at me and we shook hands. "Take it easy, kid," he said as he tossed back his last bit of whiskey, turned around and leaned over the bar waving his empty glass as a signal for a refill.

I took that as my cue to head for the exit. I had some representations to design.

is this REST?

Wed, 17 Feb 2010 19:27:00 GMT

recently, i've been contemplating the details of implementing application flow using REST over HTTP. along the way, i said in the REST IRC channel in freenode that application flows may include multiple resource interactions, some of which may be considered 'interim' resource transitions. IOW, when dealing at the application level (not the HTTP or resource level) there are arcs that involve multiple state transitions.

however, my online friend Jan Algermissen pointed out the use of interim state transitions may actually be a symptom of bad design.

so i have a simple test. below are two implementations of an online survey. i claim both of them are examples of REST-ful implementations. What do you think?

in addition, i have a number of questions at the end of the two examples to see how variations on this theme affect your answers.

a single-question survey

REQUEST ****
GET /surveys/
Host: www.example.org
Accept: application/html

RESPONSE ***
HTTP/1.1 200 OK
Content-Type:application/html

...
<form method="post" action="http://www.example.org/surveys/">
   <label for="name">Name</label>
   <input type="text" name="name" value="" />
   <input type="submit" />
</form>
...

REQUEST ***
POST /surveys/
Host: www.example.org
Content-Type: application/www-form-urlencoded

name=Mike

RESPONSE ***
HTTP/1.1 201 Created
Location: http://www.example.org/surveys/1

...
<a href="http://www.example.org/surveys/1">Begin</a>
...

REQUEST ***
GET /surveys/1
Host: www.example.org
Content-Type: applcation/html

RESPONSE ***
HTTP/1.1 200 OK
Content-Type: application/html

...
<form method="post" action="http://www.example.org/surveys/1">
  <label for"q1">Is this a REST-ful survey?</label>
  <input name="q1" value="" />
  <input type="submit" />
</form>
...

REQUEST ***
POST /surveys/1
Host: www.example.org
Content-Type: application/www-form-urlencoded

q1=yes

RESPONSE ***
HTTP/1.1 200 OK
Content-Type: text/plain

Your survey answer has been recorded.

a two-question survey

REQUEST ****
GET /surveys/
Host: www.example.org
Accept: application/html

RESPONSE ***
HTTP/1.1 200 OK
Content-Type:application/html

...
<form method="post" action="http://www.example.org/surveys/">
   <label for="name">Name</label>
   <input type="text" name="name" value="" />
   <input type="submit" />
</form>
...

REQUEST ***
POST /surveys/
Host: www.example.org
Content-Type: application/www-form-urlencoded

name=Mike

RESPONSE ***
HTTP/1.1 201 Created
Location: http://www.example.org/surveys/1

...
<a href="http://www.example.org/surveys/1/q1">Begin</a>
...

REQUEST ***
GET /surveys/1/q1
Host: www.example.org
Content-Type: applcation/html

RESPONSE ***
HTTP/1.1 200 OK
Content-Type: application/html

...
<form method="post" action="http://www.example.org/surveys/1/q1">
  <label for"q1">Is this the first question?</label>
  <input name="q1" value="" />
  <input type="submit" />
</form>
...

REQUEST ***
POST /surveys/1/q1
Host: www.example.org
Content-Type: application/www-form-urlencoded

q1=yes

RESPONSE ***
HTTP/1.1 200 OK
Content-Type: application/html

...
<a href="http://www.example.org/surveys/1/q2">Continue</a>
...

REQUEST ***
GET /surveys/1/q2
Host: www.example.org
Content-Type: applcation/html

RESPONSE ***
HTTP/1.1 200 OK
Content-Type: application/html

...
<form method="post" action="http://www.example.org/surveys/1/q2">
  <label for"q2">Is this a REST-ful survey?</label>
  <input name="q2" value="" />
  <input type="submit" />
</form>
...

REQUEST ***
POST /surveys/1/q2
Host: www.example.org
Content-Type: application/www-form-urlencoded

q2=not sure

RESPONSE ***
HTTP/1.1 200 OK
Content-Type: text/plain

Your survey answers have been recorded.

for your consideration

now consider the following:

how do you create a REST-ful implementation of a survey with 75 questions?
what should be the server response when a client application activates a question out of order (jumps from question #1 to question #13)?
does your answer change if the survey is adaptive? IOW, subsequent questions depend on the answers to previous questions?
how do you create a REST-ful implementation of a timed survey; one in which each question must be answered within a fixed amount of time (60 seconds) and/or the entire survey must be completed within a fixed amount of time (30 minutes)?
how do you create a REST-ful implementation of a survey that allows clients to save their partially-completed work and come back to finish it later?
if a client returns to complete a survey, how does the server make sure the client picks up at the same place they left off?
what if the user wants to complete the survey from a new location (home instead of office)?

Where S is a steady-state beginning...

Thu, 11 Feb 2010 18:51:00 GMT

EFF's 20th Birthday

Tue, 02 Feb 2010 16:03:00 GMT

HTML->HTTP->Hypermedia

Tue, 02 Feb 2010 10:24:00 GMT

i've been contemplating various flavors and forms of HTTP and REST over the past few weeks; all part of a master plan to wrap my mushy brain around the varying degrees of compliance and complication that come w/ crafting credible web apps (not so easy to say or do).

along the way i've bumped into a valuable scale first outlined by Leonard Richardson (The Maturity Heuristic) and also mentioned by Jim Webber (The Confusing Bit from REST). It's been referred to by different names; the one i use is the Richardson Maturity Model (RMM).

i've sometimes found it useful to express the same prinicples embodied in RMM from a different perspective; in a model i call the HHH Scale. but i'm getting ahead of myself...

RMM

The referenced presentations are well worth a look, but here's RMM in a nutshell:

Level 0

Single URI, Single verb
SOAP, XML RPC, POX

Level 1

Many URIs, Single verb
URI Tunnelling (e.g. ASP.NET, JSP, etc.)

Level 2

Many URIs, many verbs
CRUD services (e.g. Amazon S3)

Level 3

Level 2 + Hypermedia
RESTful Services

i like this model because it expresses, in just a few statements, relationships between a wide range of application types that all 'work' on the Web today. it also sends the message that items farther along in the model are 'more mature' than earlier ones; a handy value judgement when you're working to make a point [grin].

however, the RMM has a decidedly server-side tinge when describing the maturity of Web applications. the examples and descriptions employed as illustrations typically approach the topic from the POV of those responsible for implementing web services hosted on an HTTP server.

sometimes i am talking with folks that spend most of their time on the client-side of the equation. for them, talk of SOAP, RPC, and the like, are not within their usual vocabulary. consequently, i have come up with a slightly different version of the maturity model; one i call HHH or HTML->HTTP->Hypermedia.

HHH

this expression of Web app maturity highlights common views of the Web held by those who are in charge of creating the heavily scripted, Ajax-ified, DHTML-powered Web 2.0 applications most of us use every day.

the HHH model has three different levels:

HTML: devs at this level view Web applications through the prism of common HTML Web browser limits and the semantics expressed in the HTML Document spec; namely two possible HTTP verbs: GET, POST and a single progammable media-type: text/html.
HTTP: devs at this level see Web applications constrained by a basic understanding of the HTTP Protcol spec; meaning the four most often-used HTTP verbs: GET, PUT, POST, DELETE and simple data-oriented meda-types application/xml and application/json.
Hypermedia: devs at this level consider Web applications to be rooted in the notion of state transfer via hypermedia links; an idea first outlined by Roy Fielding in Chapter 5 of his dissertation. they recognize value in the rich link semantics within the HTML media-type and know there are other media-types (e.g. application/atom+xml) that reflect the same support for state transition links as a way to induce application flow. these devs also understand the importance of the HTTP spec as a starting point for building applications that not only use the full range of HTTP verbs, but also implement a powerful control data model (HTTP Headers) that allows both clients and servers to take advantage of shared caches, conditional requests, extensible authentication, and other services that make up the infrastructure of the World Wide Web.

whether you view this issue from a server-side or a client-side perspective, the farther along you go in the models the more felxible, scalable, and robust your applications become. it is also true that applications reflecting higher maturity levels are more demanding to architect and implement; not always more complicated, but usually more involved. in my experience, well-implemented hypermedia Web applications almost always look deceptively easy; and that's the beauty, the challenge, and the reward of it all.

so: how rewarding, challenging, and beautiful are your Web applications?