Common Usability Issues with Web APIs: And How Discovery Can Help

Last week, I shared that we often get asked, “Do API teams need to do discovery?” You won’t be surprised to hear my answer is yes.

I even recorded a short video about the topic:

But to really get the point across, I’m writing a multi-week series about APIs—what they are and how they work, the most common challenges that arise and how discovery can help, and several real-world stories about how API teams are putting the discovery habits into practice.

Today, I’ll be covering the most common usability issues that arise when developers start working with a new API. To adequately cover this topic, we are going to get into the nitty-gritty of the technical underpinnings of APIs. If you need help getting up to speed on these technical details, be sure to read last week’s article.

Alright, let’s dive in.

Customer-facing APIs are products. And just like any product, if we don’t do good discovery, we’ll have gaps in our offering. In this article, we’ll look at the following gaps that tend to arise with API products:

• Inaccurate, incomplete, or insufficient documentation
• Insufficient endpoint coverage or confusing endpoint design
• Limited access to resources
• Confusing error codes
• Inadequate or confusing authentication options
• Sloppy implementation of REST principles

Before we dive into the details for each of these, I want to remind you why this matters. When engineers encounter friction when learning a new API, it reduces their likelihood of having success with your product. You want customers to get value from your product as quickly as possible. When an API is hard to learn or hard to use, it takes longer for the customer to get value from it.

With consumer and B2B products, we put a ton of time and energy into our onboarding process. We obsess about helping the consumer or end-user get value from our product as quickly as possible.

This is arguably even more important for API products. APIs, once implemented, have incredible lock-in—because it takes engineering resources to switch products, customers are much more likely to stick around. If we can make it as easy as possible for developers to adopt and get value from our API products, we can often acquire a customer for life.

Inaccurate, Incomplete, or Insufficient Documentation

By far the most common challenge engineers face when working with a new API is inaccurate, incomplete, or insufficient documentation.

Remember, an API defines a pre-defined language for how code can interact with another service. If that language is not well documented, then engineers won’t know how to construct requests or interpret responses.

Good API documentation is how engineers onboard using a new API. Just like with any other product or service, if our customers can’t get through onboarding, they won’t be able to use the product or service. If an API isn’t well documented, engineers won’t be able to get value from the product.

And yet, poor documentation is pervasive. Over the next few weeks, we’ll see several instances where customers struggled to adopt new APIs simply because the documentation didn’t tell them what they needed to know.

The documentation didn’t match how the API actually worked. Critical business logic wasn’t described in the documentation. Field names weren’t descriptive and no further context was provided.

Why does this happen? Sometimes it’s due to documentation simply not being a priority. It’s an afterthought. This is inexcusable. If your API isn’t well documented, your API will be extremely hard to use and adoption will suffer.

But more often than not, it’s due to the curse of knowledge. The engineers who are writing the documentation forget that they have expert knowledge that their customers don’t have. They write the documentation from their expert point of view and it simply doesn’t work for their customers. The easiest way to fix this is to test your documentation with real customers.

The best teams write and test their documentation before they write any code. You want to learn if your proposed API will work for your customers before you build it. This is the heart of good product discovery.

I recommend that API teams write their documentation and then host paired-programming sessions with potential customers. Watch as the customer reviews the documentation. Watch as the potential customer goes through initial authentication, puts together a strategy for how to accomplish their goals, and writes their first lines of code. You’ll be surprised by how many gaps you uncover.

Only after you’ve verified that your documentation works for your target customers should you implement your API.

Insufficient Endpoint Coverage or Confusing Endpoint Design

With any product, we have to consider both the functionality that we choose to offer and how users might string together that functionality to accomplish their goals.

For example, we expect a calendar app to allow us to create new events, edit existing ones, add and remove guests, and so on. But having the right functionality isn’t enough.

We are accustomed to adding a guest to a calendar event by first selecting the event and then adding the guest to the event list. It would be weird to start with a guest and then select which event you want to add them to.

Every product has an underlying mental model of how it works. Good products are designed to match the mental model of the target customer. The same is true for APIs.

Choosing what endpoints to offer is similar to choosing what functionality to offer in your product. How those endpoints interact with each other should also match the mental model of the target customer.

Let’s continue with our calendar example. A calendar API could expose the following endpoints:

Users
Calendars
Events

When a developer wants to programmatically add a user to a calendar event, do they do that by accessing the User endpoint or do they do it through the Events endpoint?

There are two things to consider when making these types of decisions: 1) Your endpoint design should match the mental model of your target customer and 2) it should be consistent with the mental model you use in the graphical user interface (GUI) of the same product (if there is one).

Since we are accustomed to adding guests to calendar events via the event itself, we should expose this functionality through the Events API, not the Users endpoint.

When the endpoint design doesn’t match the mental model of the main product, it requires that developers work with two inconsistent mental models at the same time. They have to remember how specific functionality works in the GUI vs. how it works in the API. This is needless friction that should be avoided if possible.

Even worse is when necessary functionality is missing altogether. How do you know that you’ve exposed the right endpoints? How do you know if those endpoints support the right actions? We can answer these questions with strong discovery habits.

We can interview prospective customers to understand their unmet needs, pain points, and desires (collectively called opportunities). We can design solutions (in this case, API functionality) to address those specific opportunities. We can conduct assumption tests to determine if our proposed functionality fully addresses the identified opportunities.

But too many API teams have never heard of discovery or aren’t familiar with the discovery habits. It’s time for that to change.

Limited Access To Resources

Even if we expose the right functionality through our endpoint design, we also need to consider which attributes for each resource need to be made available.

For example, a calendar event has a title, a description, a start date and time, an end date and time, and a guest list. It might re-occur. It might include some additional settings like whether guests can see the guest list or modify the event.

Should we let developers fetch all of these fields through our API? Can they edit or delete them? These are all design decisions that will impact the value your customers get out of your API.

This might sound obvious. But in practice, API teams often get it wrong. Over the next few weeks we’ll explore several examples. We’ll look at an example where there were hidden rules about which fields could be updated when. In another example, we’ll evaluate an API that provides a Users endpoint, but fails to expose basic fields like an email address or a creation date.

APIs allow developers to fetch, create, edit, and delete specific resources. But the API only allows access to specific fields on those resources. To get the details right, when designing an API, we need to have a deep and rich understanding of what our customers are trying to do and how they plan to use our API to accomplish their goals.

Confusing Error Codes

Like documentation, error code design is often an afterthought. But this is a big mistake. Error codes help a developer understand what they might need to do when something goes wrong.

Sometimes unclear error codes are simply due to a lack of effort. Compare the following two error codes: 1) Invalid field 2) Expected an object for End Date.

The former tells you that one of the fields you sent in your request is invalid. But it doesn’t tell you which field or why it was invalid.

The latter tells us exactly which field is the issue (End Date) and what type of data it expects (an object). This description tells the developer exactly what needs to be fixed.

Sadly, the former is far more common.

Even when developers set out to define clear error codes, the curse of knowledge often comes into play. When an engineer is writing code, they are holding the full context of that code in their head. They understand all the permutations that might arise. And they often define an error code from this context.

But the customer doesn’t always share this context. They don’t have the same depth of knowledge about the product. Error codes often don’t make sense to the customer.

And sometimes, poorly designed error codes are passed along from other systems and the API designer doesn’t always have the needed context to provide a clearer message. We’ll see an example of this in one of our upcoming stories.

Inadequate or Confusing Authentication Options

Before you can use an API, you have to figure out how to get credentials to access the API. This can often be one of the trickiest steps when getting started with a new API.

We learned last week that there are predominantly two different conceptual models when authenticating with an API. There’s one model for when you need to access the API to act on your own data. And there’s another model for when you need to access data on behalf of an end-user.

Accessing data on behalf of yourself is often the simpler case. But simpler doesn’t mean simple. There is a ton of variation in how you request credentials, whether or not they expire, and how you refresh them. Some APIs assume that you’ll always be acting on data on behalf of an end-user and they make it incredibly difficult (if not impossible) to act on data on behalf of yourself.

OAuth is often used to access data on behalf of an end-user, but there are multiple versions of OAuth, there are inconsistencies in how it is implemented, and getting the user interface right for the end-user can be tricky.

Oftentimes the confusion that arises with authentication comes from a gap between how the API team thinks about the use cases for the API and how the customer thinks about the use cases for the API. When there’s a mismatch, the needed authentication mechanism might not be available.

For example, we’ll see in next week’s article that I initially had a hard time getting credentials for the Slack API. I wanted to use the API to automate welcome messages for members in my community. In other words, I wanted to post as myself (act on data on my behalf).

But the Slack API’s primary use case is to support apps for the Slack marketplace. This is a use case where the code is acting on behalf of the end-user. The only way to get Slack credentials is by building a Slack app for the Slack marketplace. This was incredibly confusing because it didn’t match my use case.

You’ll see across several stories that this is a recurring theme. Developers often assume their API will be used one way, but customers often have a different idea in mind.

Again, this is an example where if we want to develop solutions that meet our customers’ needs, we need to have a deep and rich understanding of what they are trying to do and what needs, pain points, and desires arise when trying to accomplish their goals.

Sloppy Implementations of REST Principles

Finally, developers can struggle to adopt a new API because the API doesn’t act according to their expectations.

As we saw last week, REST is a standard that sits on top of HTTP. That means there are expected norms that should be followed when you create a REST API.

These norms apply to how we define our endpoints. An endpoint should grant access to a resource. They apply when we define our methods. A PUT should replace a resource. A PATCH should partially update a resource. There are norms for how requests and responses should be structured.

But many APIs do a sloppy job of following these norms. Endpoints map to actions instead of resources. PUTs act like PATCHes. Responses have incorrect status codes or status descriptors, are missing necessary headers, or include empty bodies when it’s more appropriate to return a resource.

From my interviews, I’ve learned that developers have learned to just put up with these inconsistencies. But it wastes a lot of time. They spend hours discovering how an API works through trial and error.

We can do better. This one doesn’t even require much discovery. It just requires that we be familiar with the norms and that we abide by them.

There’s More to Come

We’ve covered a lot of ground the past two weeks. You’ve learned what an API is and how they work. And now you’ve been introduced to the most common usability issues that arise when learning a new API.

Next week, I’ll share my personal story using several APIs in my business. And then in the subsequent weeks, we’ll dive into some real-world stories about how product teams are using the discovery habits to improve their APIs.

If you haven’t already, be sure to subscribe below so that you don’t miss the next article.

The post Common Usability Issues with Web APIs: And How Discovery Can Help appeared first on Product Talk.