Web API Design Maturity Model

unedited

The talk I thought I would bring today is one that I’ve been working on this year. It’s a pretty new talk. This is the largest group I’ve actually shown this to, and it’s on the idea of a design maturity model. So you’ve already heard about me, this is how you get a hold of me. This is how you find me on Twitter, LinkedIn, and Gibhub and stuff like that and I’d love to talk with you, learn what you’re working on and what you’re doing. I always would love to do that. And as was mentioned I work in a group called API Academy. I’m really privileged to work in a group of very, very talented people that we don’t sell a product, we don’t sell a service, we actually are given the ability to research and travel and learn from people and talk to lots of people and share with you. So that’s what I’m doing today is I’m actually sharing with you things I’ve learned from customers we’ve worked with and in our own group. And as mentioned this is the book that I’m supposed to be editing now instead of being in here, so as soon as I can get myself into a chair to edit the last few pages we’ll get this honey badger out and you guys can check out this next book. I’ll use material from the book today.

So web API design maturity model. This is kind of a different approach. I think everybody’s familiar with this one, right? We’ve seen the Richardson maturity model. This is a version via Martin Fowler. Lenna Richardson’s not a fan of Martin’s rendition of this story. There’s a lot of background I don’t have time to cover but we have this notion that we start with just a real simple…back in the day it was an xml stream. We would send messages back and forth and then eventually we would have the idea of resources and then we’d think about using the verge correctly instead of always using post. And finally there was this thing where we’d start using hypermedia. And what Leonard was talking about is this idea that there’s kind of a progression going on. This is what he said. I did the Richardson maturity model because I noticed that each step corresponded to the adoption of a specific technology. So, now I’m using SOL for messaging and now I’m using the actual verge correctly. Now I’m using hypermedia text.

It would change the way we would represent the API. And that gave me a lot of ideas and over the last several years I’ve thought in a similar way. But I thought a little bit differently about it. I’ve thought about a different kind of progression, a progression where it’s the models that we use in our heads to decide what the API is going to be. And we’ll talk about each one of these. I did this because I notice that each step corresponded to the adoption of a model description. This is the model I’m sharing with you, and it turns out that’s how our APIs work with shared understanding. So what are the models we’re using and why do we use it? So while Leonard’s idea talks about the response documents that we send back and forth, and that kind of maturity and what kind of technology we’re using to send and receive, I’m going to talk a little bit about what we do in governing the actual design inside an organization. How do we describe the APIs and how does that affect the way we actually implement them and talk to each other.

So I have a couple of different things. I have this notion a database-centric design and an object-centric design. I’m going to package those in what I call internal models based on the way our internal software looks and behaves. And then this idea of resource-centric and this weird one affordin-centric [SP] which we’ll talk about at the end. And that’s more of an external model approach. How the external interface behaves. And those are different than the internal model. In other words I’m either going to share my internal object models or data models or I’m going to share my external interface models.

Let’s talk about the internal models first. These are the ones we’re most familiar with, the ones that I see most of the time when I visit with customers. So we’ll talk about the data-centric model first. Usually the API is simply exposed as the data model, what the tables are, and what the table relations are, or what the views are or what the procedures are or something like that. Our data model is the model we use for API. And this happens to be the go to approach for most IT organizations. In fact, what we find is the larger the organization the more likely they are to say our data model is our API. They spend a lot of time and effort in their data systems and their data organizations. The data is the thing that seems to last the longest throughout the life of any other technology or any other thing they’re doing. So the data seems to be the one they want to share and there are lots and lots of products and software services products that encourage you to do it this way. They simply say point us to your database, I’ll get you an API. Stand down, go have a beer. That’s basically the way it works.

So this is an example, I’ll use some different examples and I’m calling these out simply because they exist and they’re easy to do. This is a package, an NPM [SP] package. It’s simply called receivable, and as you can see this is sort of like a config file. It simply says look, here’s the database, the user, the password, the server and the database name and you’ve got a couple other instances or options. And then it says, look let’s set up some routes. So here’s my route. And you can do a gap and a customer and here’s the query that goes along with it. And it says you can do a post and then here’s the insert statement and so on and so forth. So we’re literally mapping the API, mapping the http resources and methods to the data model itself. Now there’s some pluses and minuses. The thing is there’s no design going on here right? All I’m simply doing is porting.

So this gets a zero on my scale because there’s no really big effort going on here. It’s quick and easy. It’s very, very effective. It’s also very, very optimal or efficient for internal use. Like I just need to expose some customer data to my team. Let me just do this and now you guys can have it, right? I’ve got other things to do. I got other projects. But there are some downsides. It exposes your business model and or valuable intellectual property. Often your data model is really sort of the business intelligence of the organization. So this is fine if you’re exposing it internally because we’re all in the same business, but if I’m exposing this to the outside world it could be a real problem. I used to say to people, if I’m working in a particular space and I want to find out about my competitors the first thing I’ll do is sign up for their API because they’ll tell me what their business model is. They’ll tell me their internal domain context and all their ubiquitous language and all the things that’s important to them and I can use that in my intelligence.

It’s often very tight coupling so if somebody changes a table, API breaks. So it’s a very brittle space. And it also is API may depend on a lot of sort of data tech or [inaudible 00:08:08] or having or some of these other more advanced statements that are going to be a little hard to deal with in sort of an http space. So you have to really careful. Often if you have a lot of advanced data modeling you end up using a lot of stored procedures and other things to get them to work. The biggest thing is that this pushes the cost to change onto API consumers. I just change my database. it’s easy for me. Everybody else can learn to live with it. So I love this line I’m going to quote [inaudible 00:08:37] former API Academy member and now working as CTO in healthgroup. First step in breaking the data-centric habit is to stop designing system as a collection of data and instead design them as business capabilities. One of the big challenges here is all I’m really doing is making new data. I’m not really telling you what’s possible or what you can and can’t do. You have to know that yourself.

All right, so the next sort of level in terms of design is to move away from the stored data and think about the internal objects. So your internal object model, if you’re an object oriented programmer which makes a lot of sense and again I see a lot of this in IT shops. This is very common for a classic service oriented architecture patterns or canauticle model approaches where people have these canauticle ideas about what a user is or a customer is or a product is or a server or a warehouse, or a sales rep, or so on and so forth. So this is really common in the soap style where what we really do is we already have…we know what objects are and we simply expose them through some kind of interface. So here’s a classic way of describing that interface and this is the WSDL document, the web surface definition language. And you can see it actually says there’s a service down below and that there’s this thing called request. It takes an argument and it responds, it has one argument and then you can put the two, the request and response together into an interaction, an operation, and you go on and on from there.

So we’re describing the interactions in terms of objects and properties. So there’s some design here. I can actually create this interface of this facade using a soap tool or something like that which means I can create functions. I can change my data model later and not hurt anybody. There’s lots of great tool support. There’s even lots of things that will generate this WSDL document directly from your code, so in Visual Studio or Eclipse there’s basically a WSDL button, there’s an object interface button. I punch it. It generates WSDL and it sort of purposes up my code, so really again it’s exposed pretty quickly. They can be built quickly, they can be very use case rich. I can have a function that’s called "get customers by last name if balance is over a thousand." I can be very, very specific and I can create a very valuable interface that makes sense to people when they read it.

It could be very targeted. I can create different interfaces against the same data model, the same internal model so that anonymous users or partners have a very different interface than internal users. But the thing is, again, changes to the internal models can leak out of the interface. If I start to change the number of properties on an object or it turns out my interface needs to be a little bit different, then it’s possibly going to break others. Often the consumer model is not the provider model. So my internal object model about the way I handle customers and late payments, is probably not the model I want to share with customers. So customers they need a different amount of information in the organization. So often I have this sort of friction between the model on the outside world and the model on the inside world. That means often I build two models. But at least we’re getting some sense of what’s going on. The biggest challenge is coordinating the consumer and the provider together as time changes. So it can be a bit of a challenge. So Allen Kaye [SP] taught us that messaging is the really big idea. It’s messaging, not object models that we really should focus on. So, that leads me to external models.

So the resource-centric version, this is like an http style of resources. There’s like I’m going to expose a bunch of resources for people and this is very common in browser in mobile development shops where we’re really focused on the http resource model. And there’s lots of great resource rich tools out there, swagger, open API or rammel, a blueprint, and there’s a handful of other ones that describe the actual resources first. I describe the resource, I say what’s in the resource and what you can do with it and that really becomes the model. This happens to be some blueprint here. So again this is external design. So we’re moving up on the ladder.

Now the object model and data model don’t matter. I have an external model that I can actually create specifically for a target audience. It’s focusing on the interface and often it’s very consumer-centric when it’s done well. I actually design this interface for consumers. But sometimes it’s just the internal object model turned into resources. We have a customer resource and you can do four things on it and nothing else. You know, things like submit, apply, and share and review are actions that are really confusing, so I have to create some other resource that allows those things to happen. I can still just do add, insert…you know or insert of the… It’s also very http-centric, so what happens if I need to do something over web sockets or thrift or other things. Often these models fall down because they assume that sort of magical force, that crud over http is the only way I can interact. So it can still leak internal objects and often it needs to icenmarket [SP]. In other words, if I change resources I end up breaking clients anyway. Okay, and the design patterns, the actual object design patterns reminds us that we want to focus on interfaces, not objects. So it’s a good idea that we’re headed to the interface space.

Finally last one here. This is what I call the affordance-centric, and the API is actually expressed as structured messages that have things you can do in the messages — that actually tells you what’s possible to do. So these are commonly referred to as hypermedia formats. This also happens a lot in the reactive style where even the short message format is all about what you can do, what is over the wire back and forth. So html and Adam and Hal and Siren and collection Jason and Mason and uber and there’s even more of these being developed. These are all structured formats to go beyond just curly braces and rango brackets, but actually have specific things inside. This is a link. This is a form. This is how you read. This is how you write. These are the arguments. It’s all described in the message.

So here’s a description language that’s aimed at this sort of affordance level. It’s called alps. It’s something that I and a couple other people have been working on. I and Leonard and a guy by the name of Mark Foster. So it actually describes things like this is an item and it’s a safe action which means it can be translated to GIF. Or here’s an item potent action which means it’s an update which updates a product, and so on and so forth. So it actually describes what could possibly be done and then you turn that into a format and you use that over [inaudible 00:15:15] or some other language. So it’s more about what’s possible rather than exactly how. It’s very external. It’s external of all the models. I could change my resource model, my protocols, my object model, my internal data model and never upset this contract between two people. It focuses on use cases and actions, not just the data model. It usually doesn’t restrict over protocol or workflow. You can change those overtime without breaking the contract. But the problem is there’s almost no tooling to do this.

There is no hypermedia-studio, or anything like that. Also from machine to machine cases we still have to rely on things like vocabulary-sharing, and other things because just telling me what’s possible for a machine isn’t enough for me to actually figure out what to do. So it focuses on action over data but it means that we’re relying more on shared dictionaries. Okay, so I’ll invoke Roy Fielding here and it’s an API event so I’ll do this. Roy talked about this notion of how information becomes the affordance. The message actually tells you what’s possible, that’s when it affords sales. So I got one minute. So, what does this mean? If you think about it when you have data modeling what you do is you simply think about the tables. I’ll model the tables as the customer invoice and customer visits joins and so on and so forth. So you have to understand that part. But in an object model I might create an object which is the customer summary which summarizes bits of information and now I can easily just create that summary object that has some actions like update, suspend and filter.

But if think about resources I might have multiple resources for this one activity. Like here’s the customer summary resource which links to the invoice resource, which links to the visits resource. I have this sort of rich view that I can expose and that’s not something I can easily do in object model. And in affordance model I might just say here’s a bunch of actions, and these actions might appear on every single response. Like every single page has the home or the customer view or the summary view, and I can move those responses or those actions around and actually create a kind of an experience that has nothing to do with the other model. So it’s incredibly unique. So, this is that notion of thinking about how I design things a little bit differently. So it turns out, the one thing I say, is your data model isn’t your object model. It’s not your resource model and it’s not your affordance model. You can actually manipulate each one of these differently. And that gives you some opportunities to start to rethink what you’re working on.

So I’m out of time but if you’d like to ask some questions, I’ll be around. I just want to say thank you.