What is an API?
Application programming interfaces (APIs) have been around for many years. Developers have been working with APIs in the form of Java and C# Interfaces, Python's Abstract Base Classes, and Rust's Traits. The goal is always to establish an agreement or contract through which different pieces of code can exchange data with each other. A Web API has the same goals. But rather than using the mechanisms in the programming language to enable the communication, we use (sometimes built-in) libraries to use web technologies such as HTTP. As a result, implementations of Web APIs using these technologies are often referred to as web services.
Modern APIs focus their contracts primarily on communication—the Creation, Reading, Updating, and Deleting of data (or CRUD for short). CRUD operations are commonly (but not always) implemented using a set of architectural principles known as REST (Representational State Transfer), which describes how web-based technologies should be used. When a web service uses the REST architectural style it’s often referred to as a REST API or RESTful API.
The term API has come to mean many things; API can refer to the contract (or specification), the implementation logic that realizes the API specification (the API backend), or API gateways that provide an engine to manage and secure APIs from accidental or malicious use by clients. We'll explore these different elements shortly.
Why use APIs?
The adoption of Web APIs has given us the means to decouple, scale, and integrate solutions built using many different technologies. The most extreme aspect of this has been the development of microservices. Our ever-increasing ability to build solutions from smaller parts and the explosion of APIs within organizations and from third-party services have made it necessary for us to make it far easier to discover APIs and understand how they can be used. At the same time, making our APIs easier to discover increases the risk of their misuse, making it all the more important that we protect ourselves from erroneous, abusive use or web technology attacks.
By using APIs, we can connect distributed applications. For example, we can connect a smartphone application to a social media website or a payroll system to a bank. More recently, developers have used APIs to enable the building of applications as small individual pieces that are then connected together, making them far more robust and scalable. As a result, if one piece breaks, the majority of the system can carry on (this is the basis of microservices).
APIs have enabled us to connect distributed software and use different programming languages, tools, and types of computer hardware. This freedom enables developers to independently select the best technology for each problem that needs to be solved.
The evolution in API technology and adoption has led to a suite of related API tools being developed, which have evolved into a typical architecture and features.
Perhaps the most important reason to use APIs is that a well-designed API will conceal how the API backend is implemented, allowing us to evolve and change the implementation without adversely impacting the API client. As we abstract the back-end implementation, the chances of the backend being shared (and reused) by more client applications grows, which is why APIs are sometimes referred to as SOA (service-oriented architecture) 2.0.
API components
An API management tool comprises several components. The diagram illustrates the logical interactions of the different parts involved in API management and implementation.
API specification
An API specification provides a structured description of what the API will do and how to interact with it. This is typically done using notations such as Swagger, API Blueprint, and OpenAPI Specification (OAS). These notations focus on the use of REST idioms (a way of using HTTP). There are other notations as well, such as GraphQL schema, which are more focused on specific ways (and associated technologies) of providing APIs. Older approaches to API specification include standards such as WSDL (Web Services Description Language), SOAP (Simple Object Access Protocol), and WADL (Web Application Description Language).
API designer
An API designer is a standards-aware tool that supports the creation of the API specification. It can be as simple as a plugin for an integrated development environment (IDE), such as Visual Studio Code, or you can use tools such as Apiary and Swagger, which will allow you to use rules for validating and formatting the specification.
API portal
API portals are used to find and share published API designs. Developers need to find the API specifications to understand if an API can help them and how it should be used. The portals for public APIs (i.e., APIs that are made available for consumers from other organizations and often visible to the internet) are often embedded within a website of supporting material such as legal terms and conditions. Portals are typically used to enable us to describe the rules that an API gateway applies to each API invocation.
API gateway
The gateway provides the visible URL for an API, applies rules (sometimes also called policies) to the use of that API, and then directs the API call to the back-end implementation. Typically, the gateway will be given both the API specification and details of the rules it should apply. Rules can cover actions such as:
- Authentication and authorization (for example, whether the requestor is using technologies such as OAuth, Lightweight Directory Access Protocol (LDAP), Security Assertion Markup Language (SAML), key stores, and so on)
- Certificate management, such as Secure Sockets Layer (SSL) and Transport Layer Security (TLS) termination and Mutual TLS
- Rate limiting and throttling
- Payload inspection (including payload size and the means to validate that the payload is structurally correct)
- Intelligent routing (routing based on the header or payload content)
API backend
The API backend is the name often used for the software that translates the API call into action(s). It could be an integration technology such as an Enterprise Service Bus (ESB), a database, another cloud service, a microservice, application, or web server.
In each case, the mobile app, website, robot, or computer are API clients who benefit from the use of API management.
Building applications with APIs
With the right tools, creating basic APIs is reasonably straightforward. Creating good APIs that are also secure and easy for anyone to use correctly isn’t difficult but requires an understanding of a range of subjects including application security, data design, API management, and technical documentation.
Using APIs is often made easier by using packages provided as part of a programming language or frameworks such as Spring for Java. Many of the frameworks available also happen to be open source projects. Many API providers take this further and include a Software Developer Kit (SDK) that adds logic to the API to help ensure you use it as expected, such as validating the data before sending.
Examples of APIs
There are many ways that software developers can implement APIs in their platforms. Some examples include the following:
- To access account information and user-generated content (such as pictures and articles). This can include sensitive data such as financial information and personal details.
- To allow other companies to directly connect with a specific social media profile for advertising. This is done through pre-existing APIs such as Twitter or Facebook.
As APIs have become the primary way for different software services to communicate, they have a diverse range of use cases. Some common examples include the following:
- Smartphone applications use APIs to retrieve product data from a website, such as Amazon, or map data from Google based on your location.
- Websites use APIs to update without refreshing the whole page—often done with address lookup services using your zip or postcode.
- Manufacturing robots use APIs to report that they have completed a particular job.
- Your computer uses APIs to check to see if it needs to install updates.
Here are some API examples from existing platforms with open APIs (the leading standard for REST-based API definitions) that can be explored by anyone.
- Google Maps has an API that lets programmers create new features—for instance, virtual reality tours of landmark buildings around the world. You can also use it to let people view the real-time traffic on a specific road or highway.
- Facebook has an API that allows marketers, advertisers, and developers to "plug into" Facebook's system. This allows them to create new features for users within their own applications and websites. Additionally, the Facebook app on a smartphone is just a user interface that uses APIs to communicate with the Facebook servers to retrieve data.
- NASA provides APIs to share its research data, space imagery, and event tracking. So you can programmatically follow what the Mars Rover is doing or receive data about natural events such as earthquakes, wildfires, and volcanic eruptions.
Common mistakes made when creating APIs
There are a few common mistakes when creating APIs, particularly around API specifications and how the API backend is implemented.
- The API design directly reflects the underlying data storage. As a result, if the internal data structures change, the API is impacted, which disrupts the API client. One of the goals of good API design is to abstract and protect the consumer from changes to how you implement the API backend.
- Poor documentation. Providing clear, detailed documentation is essential. For example, when you describe a date, you need to be very clear as there are different ways to represent a date (for instance, in Europe, a date is typically represented as day, month, and then year, but in North America, the order is month, day, year). Not being explicit can result in data quality issues and/or APIs breaking.
- Not considering production data volumes. During the development of APIs, the testing uses relatively small datasets. As a result, when an API is used to read data during testing, it is assumed to read all data or create/update all data. In production, the data volumes can be far more significant, resulting in API calls trying to communicate large volumes of data in a single request. This can result in a wide range of problems depending on the network characteristics between the client and backend. For example, trying to address the API request may place enormous performance demands on the API backend, which may result in the API call failing.
Mistakes may also be made during the design and application of policies for the API gateway. These errors typically involve not providing sufficient security, which can allow malicious events to occur such as users changing or accessing data or using the API as a means to start attacking the systems. These kinds of issues are analyzed and reported regularly by OWASP Foundation, with the most common mistakes being reported in their well-known API Top 10.
Confusing the roles of an API gateway and the API backend is another common mistake. Both capabilities need to process APIs as they’re received, and it’s easy to mix the two elements together. However, the gateway's job is to screen and route the requests to the right place very quickly. The API backend will need longer to process each request as it’s delivering business logic. Remember, the relationship between API calls and the API backend isn't one-to-one.
The future of APIs
APIs are rapidly evolving, and their number and nature will continue to accelerate. For example, we've recently seen APIs developed to process spoken and written language to support the growth of digital assistants and chatbots such as Siri, Alexa, and Cortana. This has been underpinned by the continued growth of artificial intelligence (AI) and machine learning (ML), with the requests and results being communicated through APIs.
As API use continues to grow and as technologies that APIs rely on continue to develop, APIs will also continue to evolve. For example, modern APIs make substantial use of HTTP, which has undergone significant changes—HTTP/2 adoption is becoming established, and the HTTP/3 specification has been developed. Fortunately, to date, HTTP has been backwardly compatible. But as the standards evolve, new API techniques are also evolving to exploit the newer standards.
New API notations are also growing and evolving. The most prominent of these today are GraphQL and gRPC, which allow us to adopt specific API styles to solve specific problems. For example, GraphQL is very good when the API client wants to choose specific data values to be retrieved, and gRPC works very well for interservice communication. Even so, the foundational principles behind REST aren't likely to disappear for a very long time.
Advanced API gateway use cases
The most common application of APIs is to provide a means for external users to communicate with back-end systems. But APIs and API gateways—along with other API management tools—can be used for many other purposes. The following are just a couple of more advanced examples.
Reverse (egress) gateways
Typically API gateways are used to protect back-end solutions from incoming requests. However, there are situations where an API gateway operates as a specialized internet proxy, providing infrastructure to direct application network traffic while allowing people to monitor what’s happening. This means that the only way for traffic to leave a network is through the API gateway, using approved external APIs. This helps with the ability to audit traffic, leaving the network with the means to better understand the traffic.
API gateways can also track the use of chargeable services, ensuring that the service is being correctly used and charged for. Additionally, gateways can help test new versions of applications before deployment. During testing, calls may need to be directed depending on whether the request originated from the live system or the one being tested. The gateway can derive the necessary information and perform the redirection.
Measuring usage, prioritizing business investments, and monetization
While monetization of APIs has been a well-documented reason to measure the use of APIs, a relatively small proportion of APIs need this. However, many more APIs can benefit from usage measurement to provide data to support the case for business and program management investments. Measuring which APIs are being used the most can help organizations determine which parts of their IT landscape demand the most significant investment and prioritize their efforts accordingly. For example, if the APIs for an aging ERP solution are getting more use than those for the CRM solution, it may make sense to prioritize improving or replacing the ERP.
API metrics can also help you better understand the kind of data being used and what is using the data. For example, if many applications are polling for master data, technical steps can be taken to ease that workload, such as caching or moving from a poll to a push model for managing the data.