Nate Barbettini – API Throwdown: RPC vs REST vs GraphQL, Iterate 2018
Announcer: Ladies and gentlemen, please welcome back Alex Salazar.
Alex Salazar: First, I love the music. Okay. First, thank you all for attending the Build track. We're really excited about this track. I hope you guys are excited about the talks that are coming up.
With that, I will introduce you to our first speaker. His name; Nate Barbettini. Nate is on our developer team at Okta. He is a developer advocate. Nate and I've been working together for what? Three years now? He is one of my favorite people. He cut his teeth at a C# developer and Angular developer so he has great experience in that stack. He's also the author of the Little ASP Book, which I'm probably misnaming but it's a great book if you haven't picked it up. He's going to be talking today about API design in this talk, API throwdown so with that, Nate.
Nate Barbettini: Hello everybody, thank you Alex. I'm super excited to kick off the Build track here at Iterate and talk about API design, which is a very fun topic.
I have a confession. I have built some pretty bad APIs myself but I'm guessing that some of you have probably built some pretty bad APIs too. Just as a quick show of hands, who here has built any type of API; big, small, public, private? Okay. That's kind of bright but it's like, most hands. Just give me a shout out, what styles and patterns were you using? Were you using REST APIs, were you building RPC APIs? What was it.
Nate Barbettini: Everybody's building REST. Okay. Very cool.
Nate Barbettini: What's that?
Nate Barbettini: GraphQL, somebody works at Facebook. That's cool. That's very cool. REST, GraphQL. Anybody doing anything with GRPC or Twirp? RPC? Okay. One. SOAP. Oh no. You got me. I actually don't have anything about SOAP in this talk. I apologize. What's that?
I thought it would be really interesting to talk about some of the big, dominant styles of API design today at a high level and talk about how they compare, how they contrast and kind of jump into this debate that's going on.
APIs in the API economy are hot topics. Everybody's talking about them. It's a big thing now. We don't always talk about API design and it's not always easy to answer the question of how should I design my API? What are the best patterns? What makes sense for my particular needs, for my particular API? This is a hard question to answer and if you pay attention to Twitter and the blogosphere and conference talks, I don't know who goes to those, you get the idea that there's this huge debate going on between the fans of RPC and the fans of REST and the fans of GraphQL and the few strong SOAP people that are left.
This is a debate that's been going on for a long time, over 10 years. It's first between RPC and REST and SOAP and now Facebook jumped into the ring in the last couple years, threw GraphQL in the mix and now it kind of reignited the debate all over again. My favorite tweet I think, that I've seen so far is, "REST APIs are now REST in Peace APIs. Long live GraphQL." Which makes me think that the author likes GraphQL a little bit.
Now, religious debates and holy wars are a lot of fun. As developers, we love arguing about browsers and Mac versus Linux and VIN versus e-Macs and all that stuff. It's fun to debate, and debates can be informative but sometimes those debates can also miss the point, I think. In this case, I think that this debate misses the point a little bit because a lot of the debates, especially recently, have tried to frame the conversation like this where we have this linear evolution or a linear progression where a long time ago we used RPC but that sucked so we replace it with REST and then that sucked so we replaced it with GraphQL and that's like, the best thing you could ever imagine, right now.
The problem with this very tempting way of looking at API design is that I think it misses the point. The point is not that GraphQL is the best thing for every situation or REST is the best thing for every situation. The point is that these are actually different tools for different jobs. If I'm building a house, I can't only use a saw. I have to have a couple other tools in my toolbox. When you're designing an API, instead of thinking about it like this, what you're really doing is making a ton of different decisions all the time.
All these different design considerations about what you want to make important or what is important to you and what isn't as important. Things like, do you care about whether your API is tightly coupled to the system underneath? Sometimes that's fine but other times you don't want tight coupling, you want some strong abstraction there. Sometimes you want APIs that are very high performance as they communicate and send messages to each other and it's totally fine for them to be totally chatty on the wire, on the network but other times, you don't want that chattiness. You want very low network latency or very low network overhead. We could go down the list.
I wanted to call this the five C's, but C5DV is not as catchy and I couldn't come up with a good acronym for that. There's a lot of design considerations. What we really should do and strive to do as API designers or API builders is understand all of the trade offs that go into various styles of API design. That way we can pick the style that's going to fit best for a project that we might build.
That's my goal today.
I'm going to talk about each one of these three major design patterns today; RPC, REST, and GraphQL, come at it from a very practical point of view and help you understand the trade offs so that you can at least have a framework for trying to decide how to pick a style or pick a pattern for building your API.
I'm not myself a Rastafarian, and I also don't work for Facebook so I try to be as neutral party as I can in this debate. So, if that sounds good, let's jump in and take a look at these three patterns and then at the end, we'll sum it up and talk about some common scenarios where I think I have an opinion on what the right style or the right pattern is, to use in particular cases.
Let's talk about RPC. There were a few people who said they were using RPC but the majority of the room said REST. I think RPC is a really inteRESTing pattern. It's been around for a long time. Depending on how you count it, it's the 80s or maybe even the 70s that this pattern started being used.
A lot of people think that an old pattern means that it's bad or outdated but I don't think that's the case. I think there are specific cases where RPC is still a fantastic way of building an API today. If you're not familiar with this pattern, it's very straightforward and simple. With RPC, the big idea is that we're calling a function on a remote server. If we take the functions that maybe already exist in our system and put them on top of some transport like HTTP and wrap it with an HTTP endpoint, we have an RPC API.
This fits really well with APIs that are very command-oriented or action-oriented because, with a function, you're calling, you're invoking something on another system.
Let's take a look at what an RPC API might look like. Let's imagine that we have a system that deals with, say, threaded conversations and messages. It might be a chat system or maybe I'm trying to beat Mr. Atwood at his own game and I'm building the next great forum software. I don't really know why I would do that. He's done it really well.
Let's say that in our underlying system, we have a couple of functions. We have a function that will list all the conversations in the system and we have a function that, given a conversation ID, we can list all the messages in a conversation and then, given a particular conversation ID and a string or some text, we can reply to a conversation and add a new message into a conversation. If we take those messages that are in the system and then wrap them up into an endpoint, we might have an RPC API that looks something like this. We have a /listConversations endpoint, a /listMessages endpoint and a /sendMessage endpoint.
The thing that's really easy to understand about this is that, in every programing language that we use on a day-to-day basis, we have this metaphor of calling a function. No matter what, if it's a backend language or a front end language, we have this metaphor of calling a function by name, passing some branders, getting a response. We literally just take that same metaphor and put it on top of HTTP or some other transport, we have a REST or RPC. I got to stop saying that. RPC API. The interaction between a client and the server in an RPC API is very straightforward and simple and easy to understand.
The client could call that /listConversations endpoint, get a response that has all the conversations in the system. It might be responding with XML, maybe JSON probably now-a-days but this is straightforward. Also straightforward to get all the messages in the conversation and straightforward to send a message or reply to a conversation. If HTTP is our underlying transport, then it makes sense to honor all of the HTTP semantics around verbs so getting or retrieving information would be proper to use the get verb but for mutating information or changing something, we should put or post because those are unsafe operations.
Like I said, this is simple. It's very straightforward. It's very easy to understand. Let's say that this system is running, it's already running in production, we have a client or job description ave that's hitting with API and and we get a new requirement. We get a new requirement for pulling down some author information for a particular message. We want to see who wrote a message. Well, so far in our API definition, we don't have any way of getting those author details but, fortunately, we can add a new endpoint to our RPC API very easily. We just need to write a new function.
Maybe that's get author details or get author information, throw that behind an endpoint and we have an endpoint like /getauthorinfo that the client can hit. It's very easy to add new functions to an RPC API and on the downside, it's very easy to add new functions to an RPC API. That can be a bit of a problem sometimes. I'll talk about that in a sec.
If you think that RPC is old and outdated and not used anymore then, there's some actually really interesting things happening in the cutting edge of RPC today. You might be surprised to know that companies like Google, Netflix, Facebook, eBay, Twitch, all use RPC internally, heavily. They use specific high performance variants of RPC like Apache Thrift, gRPC and Twirp, to do extremely high performance, low overhead messaging. When you have a large internal microsystem's service that needs to communicate between microservices, maybe millions or billions of times per day, you don't want a ton of overhead for each message, you want those messages to be really small. You want the communication to be very clear and this is actually a really good fit for RPC and that's why some of these really big companies use it for that reason.
Just to recap; RPC is super easy to understand, it's very simple. RPC clients tend to be very simple. In many cases they can be automatically generated just given a list of functions because the interaction is so basic. RPC payloads tend to be very lightweight and easy to understand, very low overhead on the network and it can be very high performance. That's the good part. The downside of RPC and definitely why it gets a bad wrap sometimes is that RPC tends to have very high coupling to the underlying system.
What I mean by that is, with RPC, you typically don't have very much of a layer of abstraction between the functions in the system and external API because in many cases, you're just taking functions that already exist and making API out of them. The problem with that is that it's very easy to leak implementation details about the underlying system into the API and then the client has to start worrying about how your underlying system is actually designed instead of just worrying about the API.
They have to worry about if there's going to be any side effects to calling a particular endpoint or even just what endpoint I need to call because I don't understand how you're naming your functions. RPC APIs typically also have very low discoverability. By discoverability I mean, how do I know even how to start? If I haven't used your API before, how do I know what to call in your API? Generally, RPC APIs just have a big bunch of documentation that talks about every function you can call but that's not always helpful either. What if I don't know which function I need to call? There's not generally a way to introspect the API or send a request and start to understand, just based on the API requests, what I should call.
Finally, one problem that is a more practical problem with RPC APIs is that you typically get something called a function explosion. There's probably a more technical term for it but that's what I like to call it. Something like this: I've worked on some APIs that definitely ended up looking like this and much worse where, it's so easy to create new functions that we just do that all the time. If the list conversations function doesn't quite return everything I need, eh, I'll just made a new function called list conversations V2 or list conversations V3 and we end up with this big, huge list of functions, some of which overlap but this is hard to understand, especially if you're coming in to this API code or you're maintaining it.
The counter to this function explosion is having a better extraction between the underlying system and the API. In other words, we could solve this problem of being too tightly coupled to the underlying system if we force the client and the server to be more decoupled, to have a stronger layer of abstraction between them and that's exactly the goal of REST. REST is the most common style of API today on the internet by far. It's been that way for a while. It's a very successful pattern and it's not even really a framework or a spec at all. It's really just an architectural style.
It was originally described in a famous thesis, famous paper on internet architecture by Dr. Roy Fielding and you kind of get the sense that maybe Dr. Fielding had used some very frustrating, tightly coupled APIs and then in a rage, he wrote this new styled called REST because REST is about as far opposite as you could possibly get from a tightly coupled API. REST has a strong goal of decoupling the client and the server as much as possible. In REST, instead of modeling functions as endpoints or functions in an API, we model resources and collections of resources and links between those resources or relationships between resources.
Modeling resources instead of functions leads to a very different style of API design. It's a very different flavor. Done correctly, this can lead to a very powerful way of decoupling clients from servers, which solves some of the problems that people have with RPC APIs.
I like showing an example just to illustrate so let's take a look at that same domain, that same problem of having threaded conversations, messages those conversations, maybe some user information and see how it might be modeled in a REST API. In a REST API, we don't have a list of functions, we have an entry point. You can imagine that, if I fire up my API client and send a request, maybe it's Postman or even a browser, I send a request to your API, just on the root of our API and see what I get back.
What I should get back is a document that describes, what can I do in this API? Where can I go? I need some metadata so I can understand how to use your API. In our domain we have a very simple, simple structure so maybe the resources we decided to model were a conversation and a message. That means that I have a conversations resource and an endpoint that matches that conversation's resource and a messages resources represented by a messages endpoint. Now, this document that I got back describes that I have this thing called a conversation and it lives at this particular endpoint and some other metadata. By the way, it's a collection, you can expect that I get multiple items back or multiple resources back if I send a get request there and so on.
If we then took our API client and just follow that link to that conversations link resource, then I'm starting to get some information back about the conversations in the API, in the system. In this case, I have an array that describes that each conversation has its own location, it's own canonical, absolute URI and the system and they also have properties of their own so title, author, a list of messages.
You can start to see that we are describing a style that is a lot more like how the web works instead of calling functions. If you imagine the metaphor of a web page that has links to other web pages and a whole collection of web pages that make up a website, that's a lot closer to the style that a RESTful API shoots for or aspires to. The cool idea here is that, without any external documentation at all, just by inspecting the requests that are coming back from your API and looking at that rich metadata, I can start to understand how to use your API without any other information, without any prior knowledge.
I could get all the messages in a particular conversation by following that link and grabbing that messages resource or sub-resource in this case and that metadata is not limited to just describing where to go, where to link to in the API. I could also describe operations. Let's say that creating a new message resource is the equivalent of replying to a message. That's how I would reply in my API. I could actually describe in my metadata, that I'm returning from the API, that if I follow this particular link but use the post method instead and pass these particular parameters, then I'm creating a new message resource.
If the resource that I was using before was a website made up of web pages that link to each other, then this resource is like an HTML form. An HTML form has a form tag and some form tags and says, method = posts and some action somewhere and if I fill out that form or interaction with that form, I follow those instructions and fill out the request of the API to create a new message resource. That new use case that we dealt with in the RPC API of needing to get some author information, which in the RPC API just meant that we needed to create a new function.
In a REST API, we may already have all the information we need in fact. In the REST API, fi we decided that there was a resource called authors and the message resource linked to the author resource that created that particular message, then we don't have to do anything else in our REST API. We already have all the resources there that we need to be able to support this new use case in the client of getting the author's information.
Let's try adding a more complicated requirement. Let's say that maybe, in our app that's interacting with this forum software or chat software, we have a new requirement that says that we want to have a new widget rendered on the side of the page maybe when you hit the homepage, that says, "Here are all the top conversations in the network." Here's the most recent reply by the most recent person that replied to this message for the top three conversations.
Well, in an RPC API, we'd probably have to build a new method that returned that exact information but, in a REST API, the client is able to do this already because of the abstraction that's available in the REST API. We already decided that all those resources are available so now it's up to the client to just say, "Oh, I need to get some resources from over here and then I'll follow some of those links and get some of these resources as well." All that information is available and that client can just interact in a new way, with the API. This is really cool if you're an API designer because it means that you don't have to know every single use case, every single thing that a client might want to do in the future.
You might build your API and then five years later, somebody does a new thing, builds a new client that does something totally different that you weren't expecting but they're able to do so because you decided which resources you wanted to represent, express in your API, built the metadata that described the links between those resources and the client can do whatever they want so that's pretty cool.
You might be thinking, "All right Nate, I've used a lot of REST APIs and not all of them looked quite like that." Unfortunately, in our industry, the term REST API has been pretty badly overloaded. It means a lot of different things to a lot of different people and that's unfortunate. I've seen sometimes, people describe the OAuth protocol for example, as a REST API but OAuth doesn't really have anything to do with resources. It's an authorization control.
A lot of people think that if I have a API that just returns JSON and it uses git, put API post, delete every once in a while, then it's a REST API but, that's not true. A truly RESTful API should express a lot more than just, "I'm using those HTTP verbs." Another problem that REST APIs have in practice is that we may start out with every best intention of modeling resources and links between resources and all that good RESTful stuff but as a programmer, it's easy to slip back into that comfortable of calling functions, invoking an endpoint on a remote server, that we end up somewhere in between REST and RPC.
We end up with something not RESTful but RESTish, kind of like a RESTful procedure call type of API. The highest ideal or the nirvana of a REST API is this thing called HATEOAS, which is a mouthful. It stands for: Hypertext as the Engine of Application State. This is a really fancy, academic way of saying exactly what I've been showing you in these examples, which is, the client and the server, their communication should completely described everything that's possible in the API. Another way to say that is, there shouldn't be any external documentation required to interact with your API. I should be able to point my API client at your REST API and get back all the information about where I can go, what I can do, how to use your API.
It's a difficult thing to achieve and even really good REST APIs today don't always do this. It's a really difficult thing but it's also really cool thing. If you have everything in the application state described in the hypertext, in the responses coming back from the API, that gives you the ability to really decouple the client and the server, really well, which allows the server to evolve independently, if necessary and the client to evolve independently, if necessary. As long as they agree on the common structure that they're going to communicate this stuff with.
As an example, web servers and web browsers have evolved quite a bit in the last 20 years, I think we could agree but because we all agree on this fundamental marketplace language called HTML, we're able to keep everything working. We don't have to create new versions of browsers that support a completely different way that server X responds versus server Y responds. If you read Dr. Fielding's thesis, he explains that this idea of decoupling the client and the server and using the hypertext to describe what's happening in the application state as necessary to build systems that are stable and have longevity over decades not just weeks or months or years but building something that can be flexible enough to evolve and grow over time so it can be a stable system for over 10 years.
Some of what's happening at the cutting edge of REST today is making it easier to model that hypertext, those responses, that metadata. Specs and schemas like HAL and JSON-API and Ion and Siren and there's some others, make it a little bit easier to replace the fact that with browsers and web servers, we have HTML, which is a common markup language that we all agreed on but with REST, we don't really have a common language unfortunately, which makes it a little trickier but schemas like HAL and JSON-API and Ion try to fill that gap and make it a lot easier to decide how to model this stuff in your RESTful API.
To recap, the advantages of REST, the big one is that clients and servers can be strongly decoupled. We have a good layer of abstraction and that means that the API has the flexibility to evolve over time and one big thing that a lot of people overlook because it's kind of implied and people don't always talk about it is that a REST API is expected to reuse a lot of stuff from HTTP, which is nice. Things like HTTP verbs of course but also HTTP caching semantics and HTTP content negotiation. There's all these tools in the HTTP toolbox that are really nice to have and instead of reinventing the wheel. REST just says, "Oh yeah, you're using those, which is really nice."
Some of the problems with REST; well, I already mentioned there isn't a single spec that everyone agrees on about how REST API should be built so even within the community of people who love REST and think REST can solve every problem, there's still tremendous disagreement about how you should structure a REST API, which is a little frustrating. It would be a lot nicer if you could just go out and buy a book that said, "Here's the exact right way to build a RESTful API for your scenario." It's a lot of work to take a step back and think about how you should model your resources and which resources to model and all that stuff.
Some of the specs I mentioned, HAL, Ion, JSON-API, can definitely help with this. The other problem that sometimes, people have with REST is that REST payloads can get kind of big, especially if you're doing this nirvana of HATEOAS, if you're returning a lot of rich metadata so the client can understand everything it needs to understand about the state of the application, just in your responses. Well, those responses get kind of big. They're loaded down with metadata and REST APIs can also be a little bit chatty. What I mean by chatty is illustrated by that last example. Remember, we added a complex use case and the client was able to do it because they could just orchestrate a number of different endpoints, hit a number of different resources, get the information they need.
Well, we had to make five or six, maybe 10 different calls to the API to get different stuff and that's perfectly fine if you have a big network pipe, lots of bandwidth coming in, that's fine but not everybody has a big network pipe and a lot of bandwidth coming. Sometimes, you're on a phone and you're in a bad 2G cell area and you have 500 latency just to make a single request, not to mention 10 requests. This isn't always a problem but sometimes it's a problem and this was one of the driving factors behind Facebook and their invention or description of the GraphQL style. I'll explain why in just a sec.
GraphQL, this is the newest and definitely sexiest thing to come around in API design in a while. This is what people talk about on Hacker News and stuff. GraphQL takes some ideas from RPC and some ideas from REST and then throws a lot of new ideas in there and we get this very different style that's really interesting and kind of cool. Instead of modeling functions and instead of modeling resources, in a GraphQL API, we deal with a query. I can say the the server, "Hey, I want information about users, messages and conversations," but I only care about maybe two of those fields on those options. I don't know about all 20 other fields you have.
I can do that. I can say, "Here's exactly the things I'm interested in. Here's exactly the fields and sub-fields and linked objects that I want. Go get them for me, bring them back and leave out everything else." The server will go do that and return exactly what you want, which is kind of cool. The reason that GraphQL can do this is because GraphQL starts with a strongly typed schema.
This is a description of all the queries you can possibly make in the GraphQL API and all the types that they return. Everything is strongly typed kind of like, if you're in the job script role, this is kind of like TypeScript for your API. You describe all the types, all those linked object types in the API and then, the GraphQL client can say, "All right, I'm going to start with query and I want this title from one thing and I want the author from a different thing, go out and get it for me."
You can almost think of this as like, sending a SQL query to your API instead of calling a function or going to get some resources. Let's use that same exact domain one more time and look at how we might model that in a GraphQL API. We would start with a query. In this case, we have one query available called list conversations and it returns a conversation object. That conversation object has some properties and we can say in this particular instance, in this particular query, we only care about the title of each conversation and then plus, in each linked message, just the text of that.
We're able to traverse down this object graph and say, "From the parent, I want this stuff. From the children, I want this other stuff," and so on. We send that off in one single request, to the server and that comes back with a JSON response with exactly the shape of the data that we asked for. If we want to go deeper, we can, we can say, "Also, I don't want just the title from each conversation and the message text from each message, I also want from each other of each message, I just want their name," and that's what we would get back.
Same use case as the REST API we're looking for, they can boast all the same problem. In this case, this is one call instead of 10 calls. If you look under the hood of an app like the Facebook app for example, the Facebook app when you open it up, it sends this big massive query off to the server in one fell swoop. It sends a big query that pulls down all this stuff up about your news feed, what your friends are up to, which other friends liked your friends stuff, all this very graph-like data, pulls it all back in one single request and uses that to populate the UI and render all the widgets in the Facebook app.
This is really nice if you're on a spotty connection, bad connection, because instead of making tons of different requests, we just made one and that's easier for the network to handle in some cases. GraphQL can also do mutations, change data so we can declare something called a mutation where, for example, I'm replying to a message, provide some parameters and then that response is also strongly typed so I have a strong type coming back like a message type and I can say, "Well, in this case, I'm only interested in the ID of the message coming back," for example.
GraphQL has some very cool advantages. It was built originally for a very niche problem to solve this network latency overhead, doing everything in one message type of problem and then eventually became a little bit larger in scope, solving some bigger problems as well. Low network overhead is still one of those big advantages and it has a type schema, which is kind of nice because we can get some discoverability from the fact that the GraphQL API, in advance, publishes exactly what it can do.
I can take my GraphQL client, point it at the GraphQL API and send a request to just find out what query's available, which is nice. Of course, goes without saying, hence the name, it fits graph-like data really well. If you have data that you need to go really far into linked relations or a graph in, this fits really well. If you have very flat data, probably wouldn't fit very well.
Some problems at the moment, with GraphQL is that GraphQL is definitely the most complex of these styles. We looked at RPC being really simple, REST is kind of like simple in theory, sometimes difficult in practice. GraphQL is more complex.
It trades off complexity for some power. More powerful, a little bit more complex. Some things like caching can be really tricky in a GraphQL API because we aren't reusing those HTTP caching semantics the way a REST API does. GraphQL APIs almost always use the post verb, which has some very specific meanings in HDTP, one of which is, don't ever cache this so you have to do a lot of custom caching stuff in GraphQL API. If you're using normal patterns and react and reducts, a lot of this can be done for you but it does get complex to do things like caching.
There are also still some best practices that are still kind of being worked on or solidified by the GraphQL community. One of which is versioning with is currently in a little bit of an awkward stage I think. In GraphQL currently, the best practice seems to be you don't version or API at all. Queries can change over time and that's just fine. I'm not sure exactly yet, how that's going to work in practice because a lot of times, you need some stability over time that old clients won't bring but we'll see what happens. Returning to our original discussion of all the design considerations that go into an API, each one of these styles; RPC, REST and GraphQL make different decisions, different assumptions about which one of these design considerations is most important.
REST trades off some great abstraction and great modeling of APIs as resources for the fact that it tends to be a little heavier on the wire and tends to be a little chattier. That may not matter for your case but it may, depends on if you're working on mobile or not I guess. RPC is very simple, very straightforward, can be very high performance but it tends to have very tight coupling so maybe not great for a strong external API or an API service.
The question that we started off with, which is; which API style is best? I think, is actually not as good as a question as; who uses my API? What is my API for? What is my API going to be used for? Let's look at a couple common scenarios and I'll give you my opinion, which you're welcome to decide whether you agree with or not but this is my opinion on a couple of common styles. Let's talk about a management API or a CRUD API. This is, a lot of times, what people mean when they just say, "I have an API." Most of these management APIs such as the Twilio API, the Stripe API, the Stack Exchange API, the Okta API, these are all APIs focused on managing objects in some system.
In the case of the Twilio API, you're managing subscriptions or messages or things like that. In the Stripe API, you're obviously managing transactions and payments and orders. In general, we have some objects that we're modeling and it's important, especially in the case of public API services, it's really important to have strong discoverability, good documentation, there's going to be lots of different clients hitting my API so in this case, I think REST is a really good fit. It fits that object model or resource model really well and specs like Ion, HAL, JSON-API that I mentioned before, make it a little bit easier to make decisions on how to model this and return good, rich, RESTful metadata.
Now, not all APIs that are public API services, not all APIs are management APIs. Some APIs are really just focused on sending commands to a remote system. A good example of this is if anybody here as built a Slack Bot, the Slack API is very command focused. It's all about, join a channel, leave a channel, send a message and the designers of the Slack API decided to model it in a very RPC style, which I think was the right decision because to model this as a REST API, we would have had to try to kind of force in some idea of having a channel resource and that would end up just getting a lot more over bloated for what this particular API needed to be. They just wanted it really small, really tight, really easy to use.
A command or action focused API can fit really well with RPC. Another thing that does fit really well with RPC, that I mentioned before was internal microservices. In a lot of internal microservices, we have a situation where we need to have a very high message rate, very high message performance, high network performance and we don't want to spend a lot of time transmitting a lot of metadata over the wire like a REST API does. Things like gRPC and Twirp are really good, strong case for microservices because in the case of gRPC for example, gRPC actually uses HTTP2 under the hood so that it can even optimize the network layer and make that as efficient as possible when you're sending millions or billions of messages per day between different services.
Sometimes, depending on your particular use case, sometimes REST makes sense for internal microservices too. If your goal is not as much high network performance but having a stable API contract between teams that maybe publish totally different microservices, then you need to make sure that they don't break the communication between them, sometimes REST might be a good choice here as well. Finally, if you're building APIs that are heavily used on mobile clients or they're very graph data focused, you should definitely check out GraphQL. It's really interesting. I think it's still a little bit early but it's starting to gain a little attraction. Even for example, GitHub, started using it for their new version of their public API too, which is an interesting experiment. I'm really excited to see how that works over time.
This is a really good choice. Especially if network performance and single message payload optimization, I guess you could say, is really important for your API. It is obviously important to say, it goes without saying, that there's no silver bullet here. I think this is why a lot of the debates can miss the point. The point is not REST sucks or RPC rocks or GraphQL rocks, REST sucks, whatever the case may be. The point is that, for every API project, there's going to be different requirements, different needs and I think, my personal opinion is that the best way to approach this is to start with a very use case based style.
Try a few small use cases with a particular style and see if it works. See if it fits your use case and solves your problems. If it does, try expanding the use cases and see if it fits more use cases. If it doesn't, try something else. The goal should always be; understand what patterns fit my particular use case best. I'm just about out of time but I wanted to leave you with a couple of things you can use if you want to learn more.
If you're interested in digging in to the REST side of things, my co-worker, Les Hazlewood, is a much better expert on REST API design than I will ever be. He's a fantastic expert on REST API design and he has some great videos up on YouTube where he just discusses REST API design for like an hour and a half. I would definitely recommend looking up Les Hazlewood on REST API design if you're interested in that.
If you're interested in the cutting edge of GraphQL stuff, check out my friend Eric Baer, also on YouTube. He has some great talks on best practices on GraphQL, how that community is starting to shape up around a lot of best practices. Another thing I really like reading is Phil Sturgeon's blog at philsturgeon.uk. He talks about all three styles of API design and when it makes sense to use one, when it makes sense to use another and what I really appreciate is that he busts a lot of myths that a lot of people tend to repeat about API design. He also has a book called APIs You Won't Hate at apisyouwonthate.com which is a fantastic read if you're interested in this.
That's about all I have today. We have just a little bit of time for Q&A if anybody has any questions but other than that, thank you so much of your time. Enjoy the rest of your time at the Build track here at Iterate. Throw your hand up if you do have a question. We have time for like, maybe one.
Audience: Sorry. What would you recommend for API documentation?
Nate Barbettini: API documentation. Interesting. Well, I'll respond to your question with a question. Is it important to you for the API documentation to be written by a human or by a machine?
Audience: Human. Human.
Nate Barbettini: By a human. Okay. Well, the reason I ask is because there's some cool stuff that you can do with things like Swagger, to try to auto generate some API docs but the benefit of that is you get lots of docs, but we all know sometimes that auto generated docs sometimes don't quite go all the way.
I don't have a strong opinion on human written docs. I think I've seen some really good API docs from Twilio for sure, from Stripe, from PayPal and a lot of places uses, like with structured text, they use tools like that to write API docs. Are you asking more like how to format them or what tools to use?
Nate Barbettini: What tools. I would definitely look at, with structured text, I personally like writing in markdown but RCP, markdown, pretty similar. Let me think.
Sphinx actually, is one thing we used a lot in building our own API docs because it has just enough power to be able to handle a lot of scenarios where you want to have rich docs but not quite so much that it's like, overwhelming amount of complexity. I would probably start with something like readthedocs.io as a simple case or GitHub Pages and then when you feel like you need a bit more power, graduate to something like Sphinx.
Awesome. How about one more. We have time for a little bit more.
Audience: One of the most common challenges when creating API, which I really hate, is define the schema. Do you have any recommendations on something that would make it go faster, especially like ... GraphQL seems exciting but then I have to do that schema situation again. What would you suggest?
Nate Barbettini: Yeah. Were you defining schemas for a RESTful API?
Audience: No. I was using Sales JS, which was the API itself but I have to do all the Java files for the database table I'm using.
Nate Barbettini: It's interesting, I was having a conversation with a friend of mine, talking about GraphQL and he said he was really excited about GraphQL because of the fact that it forces you to write a schema and it gives you a clear path on how to create your API and get to some finish point whereas: with REST, it's like a little bit more fuzzy. I don't know if I've hit the finish point yet but on the flip side, what you said, it's a lot of work to build that schema too.
I don't have any recommendations on tools per se. I think that's definitely a situation where you can over engineer it. I think I would personally start out with something really small.
If you're using a tool like Sales or something, that makes sense but a lot of times, you can get really small and just start with Swagger or even nothing. I've sketched out APIs in notepad first before I went and built it. I don't know if that answers your question but schema building is hard.
Nate Barbettini: Unfortunately. All right, I know I said this like three times. One more question and then we're done.
Audience: Do you have any tips for switching from, lets say, REST to GraphQL or would you ever recommend it? Is it more work than it's worth?
Nate Barbettini: That's a great question. I don't think I have a perfect answer for it. Any time you have a system that's already built, there's a strong, if it ain't broke, don't fix it type of thing where it's a lot of work to rebuild something from scratch and then you have to justify that you're going to have that much marginal benefit after you're done.
On the flip side, there are situations where it's like, This is so bad it just doesn't fit, so strongly that we want to rebuild it. There's a really interesting video. I cannot remember the name but if you Google it or search it on YouTube, there's a really interesting video where someone did a talk about how to ... let's say you had an existing API or any type of API and you want to put GraphQL on top of it. Do you remember what the name of that video was?
If you just search for like, "GraphQL on top of REST talk," you'll find it.
Audience: Baer did the talk.
Nate Barbettini: Baer did the talk, I should know that. The cool thing there is you could actually kind of think of it as another layer. You could say that, maybe you have your experimental level and all your thin GraphQL layer does is really, just call your old API but then over time, you can start moving people to your new one so that, instead of rebuilding all at once, you kind of do it incrementally, which I thought was a really cool idea. I'd really recommend that talk if you are interested in that.