Transcript

So that’s what I’ve been working on the last few years. I’ve released a book on designing the API and the RESTful Web APIs book with Leonard Richardson on building the servers. Also did a recent video series on design.

The book I am working on right now is called Learning Client Hypermedia [title was changed after this talk to RESTful Web Clients — ed.]. How do I build the clients that can adapt; how can I build a client that when the API changes tomorrow, the same client still works? So what I want to share with you today are some of ideas that will be in the book that’s released at the end of this year. So some of the ideas are a little rough or a little raw, but hopefully you’ll get the idea.

I have 12 patterns, or 12 principles that I want to share-- four on design, that means it affects everyone, four on the basic principles, this is usually how a service or a client should behave by itself, and then four on shared agreements. This takes in a promise from both client and server for it to actually work. So some design principles, some basic behavior rules like how you act at a party, and then how you share information with each other.

The first design patterns are listed here: PASS MESSAGES, NOT OBJECTS. We’ll share what that means. Number two is SHARE VOCABULARY, NOT THE MODEL. And we’ll talk about why that’s important. Use the REPRESENTOR PATTERN, which is an implementation pattern to help you deal with messages, and PUBLISH PROFILES which is your shared agreement with everyone else.

The basic principles, some of these you’ve probably seen before: MUST IGNORE anything that I don’t understand, MUSTG FORWARD anything I don’t recognize, provide MOST RECENTLY USED (MRU). This is an idea that Microsoft brought to user experience that we’re going to use in our APIs, one of the most recently used forms and links. And then USE IDEMPOTENCE in this process, there’s a certain process on the network, which is safely repeatable, even if you don’t know the results. This makes it incredibly easy for client machines to decide if I should re-submit even if I never got a response.

And then finally, four basic agreements. USE RELATED: What if I can’t find the link or form that I need; how can I find it? Give me all the related items. USE NAVIGATION: we know that this is another UI technique, "next" and "previous," and "submit," use this in APIs and we’ll talk about how we can do that. PARTIAL SUBMIT: accept a partial form, don’t reject a form because a few fields are missing, accept that form and give me another one, we’ll talk about why that’s important. Lastly, STATE WATCH: this is an advanced pattern that allows clients to ask servers to provide additional information so that client can decide when they’re done.

As I mentioned before, I give you a bit of caution, I’m going to show you the initial diagrams that I’m using in writing the first draft of the book. They look a little rough, and they might change, but I want to share this with you as early as I can.

DESIGN PATTERNS

Let’s talk about design patterns.

PASS MESSAGES, NOT OBJECTS

Alan Kay has this great quote that he wrote in an e-mail in the 90s, I think:

I’m sorry that I claimed the term objects for this topic; the big idea is messaging.

— Alan Kay

So Alan Kay, the person who helps us understand object-orientated programming with small talk and squeak, is telling us that objects are not the important part at all. It’s the message that we transfer between individuals. And that’s really important for us in building machines, bots, or client applications that can adapt over time. They don’t memorize objects, they memorize messages. The big idea is messaging and that’s the way we’re going to design our system, it’s going to be based on messages. If you think of HTTP, HTTP doesn’t have any methods. GET is not a method, GET is the first line in a message, it says G-E-T space H-T-T-P wack 1.1 right? And then there’s all this other information. So HTTP really is a server, URL, a server domain and a port number; that’s all it is. And we pass messages into that port number over and over again, that’s all we do. The big thing is messages.

So if you think about it this way: I have a lot of objects in my domain, whether I’m working on accounting or user management or even managing APIs, and I collect up those objects to create a message. “Here is the list of users; here is my message that explains all of the accounts that are still unpaid,” and I send that message to the client. I can’t send an object, I can only send a message. I can’t send a row from the database, I can’t pull it out and send it, I create a message that represents that row. The first one is to use messages, and there are actually several message formats designed specifically for creating these hypermedia services. HAL, the Hypertext Application Language, is actually the most popular right now, Amazon uses HAL in several of its new APIs. Collection+JSON is this design I created in 2011, and it’s used by Nokia and several other organizations. Siren, created by Kevin Swiber, who works at Apigee and they use it in their IOT system. UBER is a design that Leonard Richardson, Mark Foster and I are creating, and ATOM is one that we’ve seen for a long time. ATOM is probably around 10 years old. All of these are based on messages, not on object models. If we think of HTML: I don’t send you objects but I can do a whole accounting system — because I send you messages.

SHARE VOCABULARIES, NOT MODELS

So Roy Fielding, who’s the designer of the REST pattern — the REST architectural style — says this:

It’s easier to standardize representation and relation than object and object-specific interfaces.

— Roy Fielding

What does he mean? It’s easier for people to agree on the message formats and the words they will say rather than the actual objects that they use to create them. Here’s a good example: in language, if I’m trying to learn to speak Portuguese, I learn the vocabulary. I don’t memorize a paragraph from some document and then go say that to someone. A paragraph is like an object, it arranges the vocabulary in a certain way. An object arranges all the data points in a certain way in a hierarchy or relationship.

When we speak to people we’ve never met, we often the use the same vocabulary but we may actually have different models in our head. When I say,“Oh, I’m going to pass you the user record,” you may think, “I know what a user is,” but I may have a different idea of what a user is — but we can still communicate.

So by focusing on the vocabularies, both client and server can build their applications synchronously; I don’t have to wait. We don’t have to share the same exact object model on either end of the world just to do a simple update for a screen.

So if I take the same thing that we’d worked on earlier, I have all these vocabulary — name and address and size and part number and status and color and count. These make up the person and the widget object on my service. Because my service has this idea about what a person is and I can turn that into a message, say, using Collection+JSON and pass it to a client. The client actually has this object called thing and customer. They use the same vocabulary, they place those words in their own object model, and then they can share back and forth. We can change our object models. Ee know this from refactoring, right? In a refactor we have a facade and we can change the model behind it. The facade is the API, the facade is the message pass.

By focusing on vocabularies, we have a better chance of sharing something and actually make changes on our own side without breaking yours. There are lots of great contexts for vocabulary, via a [IANA] link relation set, the schema.org site, microformats, dublin core metadata, the new activity streams from James Snell. All of these are shared vocabulary sets. What we do when we work on our code is we actually make sure that the messages are written using these vocabulary terms. That doesn’t mean our database uses these, because our database is separate and I can use a different object model in my database, but it means my messages use the shared vocabularies.

THE REPRESENTOR PATTERNS

Number three: the representor pattern. This is really a variation of one of the original patterns from the Design Patterns book from the Gang of Four — the original Design Patterns book based on the architect, Christopher Alexander.

A Strategy pattern lets the algorithm vary independently of the clients that use it.

— Gamma et al

We can have plugins. We can use a strategy that says,“I want to send this object model to Mike, the client, but I need a strategy to use a message format that Mike understands.” We share message patterns, right?

So what does that look like? This one [image] a little messy, I apologize, I wasn’t even really sure. We have this sort of standard format for the way we use objects, and then we have a strategy pattern that says,“Send the object model into the representor”, and that will use plugins to convert that object model to HTML, or convert it to Siren, or HAL, or Collection+JSON, or UBER, or CSV, or PDF, etc. Now we have this idea of this facade, which lets us refactor all sorts of things behind us, and that sends out all the different formats that we might need.

Now it works the same on the other side, the client will accept that UBER format, and use a Representor or Strategy pattern that turn it into that client’s object model. This is a standard pattern that we can use. Implementing this standard resource model or this standard internal model is that first step.

This can be a bit of a challenge because many of us think the resource is the object. Now, the object is the object, the resource is the resource, the representation message is the message, right? We have different layers. We have this resource model, and then this dispatch strategy pattern for what we can send out. And what does that look like? Here is a simple example that I use time and time again. You can see that I get an object model sent to me, I decide what format they’ve asked for, and if it’s JSON, I convert it here, if it’s Cj, here, HAL, UBER, HTML, etc.

So I now have a very easy time to create a new strategy algorithm for a new format, and I just plug it into this list. I don’t have to change the way I represent a resource. I don’t have to re-test all my service code, I just need to go ahead and implement a new strategy. This also makes it relatively safe and cheap to add new formats in the future. Many of us have a challenge when we start working on the web, we must decide which format to use. Now, we have to start with one. And if you use a Strategy pattern, adding another one is cheap. If customers don’t like the one you started with, you can add another one, you don’t have to change and rebuild your code, you just need a new strategy.

PUBLISH PROFILES

The last of the design elements is called publish profiles, and I have a quote from Mark Foster who’s one of the co-editors of the application level profile semantics or ALPS specification:

Profiles provide a way to create a ubiquitous language for talking about APIs for both humans and machines.

— Mark Foster

Now he used sort of an unusual phrase-- ubiquitous language. This comes from Eric Evans' domain driven design. So Eric Evans' idea is that we should all be using the same language when we talk about our interface, and of course that language is the vocabulary that we saw earlier. A profile specification is a machine-readable version of that vocabulary that includes not only the data points, but also the actions.

You can publish profiles by defining all the possible data and actions using the RFC 6906 rules for actually publishing, and they have a profile specification. And then servers publishe the profiler as a URI, and clients can validate that. Now in advanced cases, clients could even negotiate for a vocabulary; for a profile. “Could you please give me version three of the accounting profile?” And some servers might do this the way they negotiate for message formats. Now I’m negotiating for semantics; for domain specifics.

So how does that look? This is a sample profile document that actually lists — it may be difficult to read — starting with the transitions that this is the product transition, this is the one that gets you the collection, this is the one to get a list by IDs, this is how I edit, this is how I create. This is a machine-readable specification for all the possible things we might talk about.

One of the times someone explained this to me and they said, “When I create a web server, what I’m really creating is a way to talk about a subject.” We might have lots of new conversations, we can use vocabulary to create new sentences, so if we both share the profile, we both have an opportunity to start new conversations and this makes it highly adaptable — means that I can introduce screens that have inputs on them that they didn’t have yesterday because those inputs were part of the vocabulary.

BASIC PRINCIPLES

So we talked about the design ideas. Those design ideas give us a foundation for creating applications. Let’s talk about the actual runtime, the runtime ideas. We have some basic principles here.

MUST IGNORE

The first one is called: must ignore. That’s actually quite old; comes from some of the original XML designs from the early 90’s.

The goal of ‘must ignore’ is to allow backwards and forwards compatible changes.

— David Orchard

One of the key ideas of creating an adaptable set is to ignore the things you don’t understand. This is what I do when people speak in Portuguese to me. Right? I catch a few words. “Sim! Oh, I know what that one is. Olá, oh yeah.” I ignore the rest; can get me in a lot of trouble. But we all do this, right? This must ignore idea is very important.

So, the rule is: Clients must ignore data or inputs that they don’t understand. That may mean that you don’t render them on the screen. That may mean you render them but you don’t offer any option. If there’s a new action, a new control that’s not part of the vocabulary, you probably don’t render it, right? Because I’m not sure what’s going to happen, but at the same time I don’t reject the message. I accept it, I do what I understand and then I pass it along.

So that looks like this: I have a message that has A, B, C, and D in it, but it turns out, I don’t understand C and D, so my client is going to render A and B. That’s fine. I don’t ignore the message simply because it has things in it that I don’t understand.

MUST FORWARD

And the corollary, the other side of the same idea, is to say, Must forward. And this comes from RFC 7320, the http specification.

A proxy MUST forward unrecognixed header fields…

— RFC 7230

Proxies, or anyone that’s in the middle of a conversation, must forward on everything that they got to the next party. They can’t take things off and then hand off just part of the solution.

So when I’m in the middle of a conversation, if I have a translator who’s speaking to someone in Portuguese, and I’m speaking in English and there’s a translator in the middle. She can’t leave out parts. Right? She has to pass along all of the information from both of us. And it turns out, especially when we’re building distributed systems on the web, often we are the proxy. We’re the in-between. We don’t know how many servers there were ahead of us or how many other clients there were behind us, right? We’re just speaking in this one conversation.

So what’s important that we forward on things, even if we are not processing them. You must forward unchanged and the input fields that the client doesn’t recognize as well as any data, so if there’s a field that’s in a form that I don’t know about, I’m not going to fill it in, but I’m not going to strip it out, it may already be filled in with a default value.

What does that look like? I’ve got the same A, B, C, D, I processed A and B and when I send it to the server, I send C and D along. I just send it right along with the rest of it. I may have ignored it on the screen, but I forward it in my response.

MOST RECENTLY USED (MRU)

Here’s an interesting one: provide MRU or most recently used. I took the line from Wikipedia because I couldn’t find the original Windows documentation. I’m still looking for the original Windows documentation.

A feature of convenience allowing users to access last used files or documents.

— Wikipedia

Microsoft Windows introduces this idea. There’s some dispute if it was Microsoft Windows or Apple that first did it. But this idea — most recently used — is very important for services.

If my client application is supposed to look for all of the accounts that have a balance of over $1,000, then I keep making the same request over and over again. Services should send me that request link automatically so I can use it again very easily. It should help my machine out just the way we help humans. Services should return the most recently used links and forms in all responses, just like we do on a website, where there’s always the homepage link, or there’s always the about link, when we’re doing a service. When I’m managing accounts, or I’m processing some information for someone, I should get the most commonly used links back to me.

Now in an initial design, you can simply decide, “I will always return the home link or I will always return the create link.” But you can also advance your service to actually customize that return set based on the logged in client. “This client always does updates, so I’ll return the update link in every response.” “This client always does the filter searches, I will return the search in every single response.” And you can create services that actually adjust the response based on the logged in user, and that user can be a machine, it doesn’t have to be a person.

So this makes it very easy. What happens is if I’m doing ‘search’ and ‘add’ quite a bit, then when I get the response, ‘search’ and ‘add’ will be placed at the bottom of every response. So my machine that’s trained to look for the search link will find it very easily, because that’s how we will build client applications. We’ll build them to look for the link or the form that they want to find. That’s the way they will work, just the way humans work on the web.

USE IDEMPOTENCE

The last of these basic principles is called: use idempotence. Do we know this word, ‘idempotent’? Yes? Very good. It’s actually a mathematics term that is also applied to protocols like HTTP and FTP, CoAP and MQTT:

Can be applied multiple times without changing the result beyond the initial application.

— Wikipedia

It’s like SQL UPDATE. If I do a SQL UPDATE on record number three to field number seven, and I update that to be six, even if I update that 17 times it will always still be six in that field. That’s idempotence.

In HTTP, PUT and DELETE are idempotent. POST is not. GET is idempotent — it’s read-only but it’s idempotent. I can repeat the GET many times. So I actually show a quote from the HTTP spec — there may be some better sources of this. I’m looking for a math source. “All network requests should be idempotent, in order to allow clients to safely repeat them even when their response is unclear.”

If I don’t get a response back to the server, how do I know it was completed? How do I know it’s safe to try again? I think we all have this experience when we write comments on a blog. I write a comment, “Oh, you’re wrong.” If someone’s wrong on the Internet I must fix that, so I write my comment and I press enter, and the comment doesn’t appear. So I write it again, “You’re wrong, you’re wrong,” and I press enter and now two of them show up. Right? Because the first one did actually get there, I just didn’t see it. Now I look like an idiot. So idempotence makes it harder for me to look like an idiot, as long as I keep typing “you’re wrong” I will be an idiot, though.

What does that look like? I’ll do it over here. So I have my client that’s going to send some information about A and B, but I never get a response, I don’t know what happened. Since I know this is idempotent, I can send it again, and I get my confirmation response and I know I’m okay. Humans do this everyday, right? We try to make, “Oh is it okay; do you think I should submit my credit card again?” Maybe not. But we need to make it easy for machines and bots and components, and using the idempotence pattern will make it easy for us to do that.

SHARED AGREEMENTS

Okay, so that’s the four basic principles. Next, what we have is this notion of shared agreements. Shared agreements are things that it takes both parties to understand before they work correctly.

Let’s start with the first one. So use related, this is similar to the MRU idea, but it requires an agreement on both parties.

Services should return a related link that responds with all the possible actions in this context.

— Amundsen

I may get the most recently used response but now I need something else. I need to understand “How can I update just some of the records"”, or “How can I delete just the one that I last did?”, and maybe that’s not on the list.

So what does that look like? Basically I have all of these possibilities on this service, on how I might be able to manage an account, but I’m only going to send the most common ones, it’s like giving someone the list and the UI of all the possible things they can do, right? That’s a terrible user experience, because that’s confusing to the user, so what I do is I send the most common set. So the service sends the most common actions to the client.

But the client says,“Wait a minute, I need an action that’s not on this list. Where do I go to find it?” How do we do that when we’re humans? We get to the screen. I don’t see what I want. I start clicking around, ``Is there a list or an `abou’t or a site map? We’re looking, we click around.

This is what our machines are going to do as well. They’re going to start looking around. How can we make it easy for this machine to adapt to changes? We need this idea of a ‘use-related’ link, a single link that says “Click here and I’ll give you all the possibilities.”

This is an agreement between server and client. “I’ll send you the list but I’ll also give you the idea of the related. If you click on related, you’ll get the entire list that’s valid for you right now.” Services can be very smart about what they send. I’m the administrator and I’m logged in, that list of possible actions is very long for the administrator. You are logged in and you are a guest, when you say related, that’s a very short list — there aren’t so many things you can do here.

Services can adapt what they send based on the context of who’s logged in, and that who can be a bot. It can be a bot acting on my behalf, I can set up a service bot that starts up every morning, pulls down all the people who called in last night, asking questions about sales and makes entries into another system so that it’s all in sales queries, it’s all in my support folder or whatever I want to do. I have a bot that does that every day. And as I’m logged in as User it will act as me, but if someone else uses the same bot, it will act as them as an administrator or as a guest.

USE NAVIGATION

The next one is another one that’s borrowed from user experience that we can implement for machines. It’s called use navigation or sometimes called use next.

To achieve a single goal that can be broken down into dependable sub-tasks.

— Design Patterns

We use next and previous. We know this in user interfaces. We know this in human interfaces where we have a very complicated list of things to talk about and it’s very intimidating to have a single form where there are 25 things to fill out, and you get one wrong and it’s all confusing. So what do we do for humans? We break those off into tabs, right? Just like we see here.

We break it off into small tabs or sometimes this is called the wizard pattern, right? There’s a wizard that walks us through some tasks. And each step along the way, smart services save the contents so that if I finally get five steps in, and I come back tomorrow, it will say,“Oh, do you want to pick up where you left off?” and I can start there. The phone rings, and I get distracted, I can log back in and it still has my work for me.

We can do this for machines as well. This is an incredibly powerful workflow support for machines. Machines don’t have to memorize all the steps, they just have to know that there’s a next link that appears on the page, and if there’s a next link on the response, I can activate that link and I can go to the next step, so I can go next, and next, and next.

But, there’s more to it than that. The rule is that services should provide next and previous link, but they should also provide cancel, and restart, and done. This is what we do as humans in our heads as well, right? I get through a process and I realize, “Ugh, I really don’t want to do this; I don’t want to share my credit card, I don’t have all this information. Cancel my order.” We need to provide that cancel link.

Second, we may want to restart from the beginning. We may actually get confused and suddenly realize, “I made a choice, I made a wrong step, I want to go back and start again.” And third, there should be an ability to say done. Like even if I’ve only gone through five steps, or I’ve gone to the first one and jumped ahead and so on so forth, at some point I’m done, and I just want to push send, or submit or whatever the option is. This is an important pattern to implement for machines for services and clients.

What does that look like? It looks pretty simple, I have my first screen which has a next on it, and maybe now it starts to give me a previous right, I don’t have previous at first, then maybe it gives me a cancel idea and I move to the next step, maybe there’s a restart or a quit or done somewhere along the way that leads me somewhere else. Now I can write in an application that understands this agreement of next, previous, cancel, restart, done. It will work with any server that uses the same pattern about any subject, about any thing. I don’t have to write a bespoke client for any particular workflow. I can write one for a general workflow, which is exactly what we do for humans, right? When I go to your website and you have a workflow, I don’t have it memorized ahead of time, I follow instructions along the way.

So by creating a pattern like a shared agreement, like the use navigation, I can build a client that can do this on its own.

PARTIAL SUBMIT

So then there’s this notion of partial submit. This idea comes from a UX designer, a user experience designer by the name of Donald Norman; has anyone heard of Donald Norman? Donald Norman wrote a book called The Design of Everyday Things. It’s a great book. Tt’s an easy read and you should read it. All of us who work in any kind of human-computer interaction, this is a really interesting book. He actually wrote the original book in 1988, and wrote a new edition in 2013.

And it talks a lot about why devices like this little handheld device works so well, this has no instructions with it, it didn’t come with instructions right, but it comes with four buttons, and I understand immediately what that’s like. It’s a user interface I can understand, that I can learn quickly, that I don’t need a manual for. Most of our APIs that we design come with instructions because they’re really complicated, but there are ways to design APIs that look more like this, that I can understand without instructions.

Think of all user actions as approximations of what is desired.

— Donald Norman

So anyway, one of the ideas he has is to think of any user action as an approximation of what they want, to think of a user trying to get something done, even if they make a mistake. Instead of rejecting their attempt and saying, “No, you’re wrong,” say, “Okay, thank you, but you’re not quite right; can you give me a little bit more information?” This is what good instructors do, right? We know this experience in real life. When someone who says, “Can anybody answer this question?” and then you answer it and they say, “Yes, but that’s not quite right, do you have some more that you can explain?” So they get the right answer from you. And we all have experience with instructors who just say, “No, sit down, you’re wrong!” That’s not very fun for us, right?

So Norman’s idea of treating a user’s actions as an approximation, as a way to get closer to success, is behind this idea of partial submit. Partial submit says services should accept partially filled in forms, ay “yes, 200 okay”, and return another form with what inputs are remaining. So if I have a form with five fields in it and I entered three and I pressed submit, then services should say, “Okay,” and give me a form with two fields in it now, the two that I missed, and keep those three for me later. This is a variation of the wizard option, right? I can have this experience back and forth.

Now as a human user, this can be rather annoying and tedious, I keep filling out a form and it keeps sort of changing its shape, I never know when I’m done. But for machines, this is actually very efficient and very effective, because machines can just look at what they’re being asked to do, and give it to you.

In fact, in several experiments, I’ve been writing clients where the forms only have one input, so the service says, “Give me this piece of information,” and you go to the next thing, “Okay, give me this other piece of information,” then you go to the next thing, and I can actually lead the bot around to provide all the information I need. It turns out for bots, having lots of options can be confusing; having one option is easy.

So what does this look like? I have a service and it gets a form with three fields in it, I submit it, some of the clients submit two of them, and then the service sends back one more field waiting for me. Now it’s important in this pattern to also provide a cancel, or an escape, because I might have a bot that suddenly gets stuck, keeps getting asked for one bit of information that it doesn’t have, so that bot can say, “No thanks.” So often this pattern has a cancel and a done, or a cancel and a submit. The cancel says, “I give up,” the submit says, “Look, I just want to send it the way it is, I’m done, I have nothing more to offer you,” and the service can decide what to do. So partial submit and use navigation are both ways to help someone fill in data; help a bot fill in data safely. Even when that bot doesn’t have all of those arguments memorized ahead of time.

STATE WATCH

And that leads us to the last one on my list that I wanted to talk about, which is very important, and that one is called the state watch pattern. State watch actually comes from a gentleman by the name of Jens Rasmussen who is in Scandinavia, and he uses this idea that there are data elements that we watch in order to decide what to do next — in order to determine how we are going to interact with the world.

Data representing variables in a dynamical system.

— Jens Rassumussen

In fact, he says there are three things. There are signals, signs, and symbols.

So a signal is maybe the temperature gauge on some pressure device. Is it warm? is it hot? That’s simply a signal, and a signal is like, we just see it.

A sign is when we interpret that signal to mean something. “Oh, if it goes up on the scale it means it’s warm; if it goes down on the scale it means it’s cold,” that’s our interpretation, or, that’s our sign,

Symbols tell us what that thing means, signs tell us what we should do about it. If this temperature gets above this level, that’s a sign that it’s too hot. That’s a sign that I need to take some action. Right? This idea of signals that we take in, symbols we interpret them, and signs give us an idea for action; comes from Jens Rasmussen. When we’re working with adaptable systems, we need that same idea.

This is Jens' example using the temperature model that I talked about. In the first case I have a signal which tells us what that number is, in the second case I have a sign which explains what it means, if it’s above C, it’s okay, if it’s above D then it’s not okay, and then finally I have a symbol which says if it gets down here then something is wrong and I need to figure out what this means. This symbolizes something.

So, what does that look like? Services should allow clients to subscribe to watch values, allow clients to say, “Look, I want to watch the temperature.” or “I want to watch the total count of users.” or “I want to watch the total backlogged orders in the queue.” Services should allow clients to subscribe to that so clients can determine what action to take; so clients aren’t just there to do as they’re told by the services, clients are there to interact with the ecosystem and they can decide what next step they’re going to take.

So what does that look like? Now I have a client application that’s watching the temperature and waiting until it gets to some level, Z. It has some point, if it gets equal to or above Z, then it will take some action. So maybe there’s an interaction over and over again between the service and the client, maybe the client is turning up the temperature just a little bit because I was told I needed to keep it above Z, keeps doing it over and over again and asking, “What is the temperature? What is the temperature? What is the temperature?” and finally, gets a response that says, “Oh, it’s now at Z,” and it knows it’s done. It’s exactly how the thermostat works to decide how cold it should be in this room, right? It’s watching a variable over and over and over and when it gets within a certain range, then it says it’s done.

Now there are lots of variants on this story, one is that the value you’re looking for is very specific, it must be exact, that’s usually not very workable for machines, usually the value’s in a range, as long as it’s 10% up or down of this target, it’s good enough.

Another thing that may happen is that client applications may memorize their watch value. I’m a temperature sensor, it’s all I do; I memorize, there’s a temperature value and I want that in my response. But there are other clients that might be able to adapt. “I’ve looked at the vocabulary, I’d like to know about temperature sensors and the number of people in the room, because I will change the temperature based on the total number of people. If there are many people giving off a lot of heat I will actually lower my target temperature right away.”

So there are clients that can actually actively subscribe, or get it from a configuration set, so they’re a much more generic client, or there’s a client that’s specifically built for one or two variables. This is exactly how the Internet of Things is going to work. We’re going to create devices that pay attention to signals, and understand what they signify and what they symbolize. This is all a chance for us to start building those things today.

SUMMARY

I have 10 minutes. Something like this. Okay, I think we’re doing good.

So, we’ve talked about these 12 patterns, we’ve talked about them in three different sets, this idea of design, principles, and agreements.

DESIGN PATTERNS

The design patterns are sort of big picture items; design patterns tell us when we design a system and implement the basics of the system, 1) we’re not going to share objects, 2) we’re not going to require that everyone have the same model of the world that I do as a service or I do as a client. 3) We’re going to use the strategy representor pattern to make sure that as things change over time, my code doesn’t break, because I’m really just going to memorize message patterns rather than the actual object model. And 2) we’re going to use profiles to create, especially for machinery, you have to create a profile that limits the amount of vocabulary and actions I need to understand. As a bot, you can’t introduce a new piece of vocabulary that I never learned, it’s like starting to speak a new language to me, I will just not work. So we need to create these language boundaries. I’ve met a profilist, Eric Evans gives great guidance on domain-driven design and Vernon Vaughn has also written books on domain-driven design, they all give us this idea of publishing profiles.

BASIC PRINCIPLES

Once we have these design patterns, we have some implementation principles for all of our applications, and that is that we 1) must ignore things that we don’t understand, we can’t reject a response because it contains something that we’ve never seen. What does this mean? It means that I cannot use schema to validate an incoming message, because a schema is often very strict, right? McIlroy, one of the people who invented the Unix platform has this phrase: “I will be strict in what I send and liberal in what I accept,” and that gives us one of the authors of the TCPIP specification. I will be very strict, I will make sure that the messages I send are as close to the specification as possible. But when you talk to me, even if your English is not so good I’m not just going to walk away, I’m going to do my best to understand what you mean. So that’s what must ignore is about.

2) Must forward, this idea, that I must forward on things that I don’t recognize. If I’m standing between two people, and one person hands me this document and I understand some of it, and I’m going to hand it to someone else, I don’t erase the things I’ve never heard of before and then hand it to somebody else. This would never work in a network, right? I simply pass along what I received. Now I might modify the things that I understand, my job is to update the discount value on all of the accounts, but I’m not going to mess with the rest of the information and then I’ll pass it off to the next person and she can work on it. 3) Provide MRU, make it easy, so that I don’t have to look for things that are important to me as a client, add those items, and 4) Use Idempotence, make it easy for me to send a second time in case I get confused or don’t understand so that it doesn’t hurt the system; make that possible.

SHARED AGREEMENTS

So those are the basic principles and that works whether I’m a provider or a consumer. Now the agreements are a little bit different. The agreements are sort of advanced patterns right? The agreements start with 1) use related. If I can’t find the item I’m looking for, give me a link that will return all the possible items, show me the index, show me the sitemap, and I can do that dynamically based on somebody’s login.

2) Use navigation or use next: break things down into dependable units-- Tab 1, Tab 2, Tab 3-- so that I can take them one at a time if necessary. By the way, when I do this use next and the next one partial submit, this gives the server the possibility of actually changing the path that the client goes in based on their inputs. I have more freedom, a client doesn’t have to decide as many things, I simply need to provide the information that we need.

3) Partial submit is this idea from Donald Norman, this idea that I want to accept whatever you give me even if it’s not quite exact and just ask you for the rest. I won’t reject a partially filled in form, I’ll take whatever you have and then prompt you to help me for some more.

And finally 4) the state watch pattern from Rasmussen, this idea of using signals as a way to determine what I’m going to do next. I will ask you to provide me with a temperature signal, provide me with a queue count, provide me with the last known address of this customer; all these things I’m asking you to supply to me so I can decide what to do next.

Each of these things along the way, the principles and the agreements as well as the design elements give us a framework for starting to build things that can adapt over time. If the order of things change, if the exact inputs are different, if the links that appear in one page are different than all the possible links, my same application can still work. This is essentially the way micro services are going to operate this world of discoverable components. These are the principles we need; I can’t build individual components that only do one thing that they will memorize and expect them to last a long time, because our rules change, or domains change, so I need to build components that can adapt to some of those changes.

MAKING CHANGE EASY

And as a final quote, the thing that I’d like to pass along, this is a quote from a gentleman by the name of Paul Clements. Paul Clements worked at Carnegie Mellon University. He’s now working in the private sector in the United States. He has this great quote:

The best software architecture knows what changes often and makes that easy.

— Paul Clements

Makes those changes safe, relatively inexpensive, resilient. And that way, as things change over time, my system can adapt in small ways. Adaptation doesn’t happen in leaps, it happens in tiny, tiny ways. Paul Clements made this quote, I think, 15 years ago. This is really what SOA is based on, right?

This idea that we can create an architecture that knows what changes or what adapts, and by taking on these 12 principles, and by having this attitude about knowing what can change over time, that gives us the chance to create the adaptable clients and the evolvable APIs that we could be looking for. And with that, I just want to say thank you very much for staying with me, I hope it’s very useful and interesting.

Thanks.

QUESTIONS

Mike

Do we have time for questions? Okay, yes. Do we have questions? Anybody would like to ask a question? We have a microphone right here, yes.

Person

Hello, great talk. First of all, congratulations! I was [inaudible 51:08] about your opinion about some of these [inaudible 51:12] pacts that you can see around for APIs, I don’t know if you are aware of things like jinx in API, that they try to address that in a simple way and try to set a standard on how to do that sort of thing. I would love to hear your input on that.''

Mike

Yes, I think the question is about-- There are a lots of other standards than just the ones I listed, right, JSON API comes from the Ember team and the Ruby team, there’s OData from Microsoft, and there are some new ones coming on. I have mixed feelings about some of them, I didn’t even mention RDF, right, there’s a whole series of RDF formats. Usually I tend to use the formats that give me inline instructions like forms and links, but some of these formats don’t do that. OData and JSON API sort of have rules about how you construct your URLs, with OData there’s a whole [inaudible 52:00] language, with JSON API there’s a relation item and a few other things. I tend to not like those because they sort of restrict what I can do, but I think what’s really important is to treat these as formats rather than protocols and I think that’s where some of the formats make it a little tough, it’s hard for me to actually implement JSON API as one of the items because there are so many sort of built-in protocol agreements like the ones I mentioned here. But there are many more and I suspect there will be more, I think we’re in a kind of explosion of formats and eventually we’ll probably get down to some like HTML or CSS, but actually when HTML came out there were about four other competing formats we sort of forgot 25 years ago. So I think we still have more work to do in this space, and I actually want people to use as many as they can. Does that help? Yes, any other questions, anything? No?

Well then again, I thank you very, very much and I’ll be here today and tomorrow, so thank you.