API Monitoring: A Complete Introduction

At the most basic level, application programming interface (API) monitoring checks to see if API-connected resources are available, working properly and responding to calls. API monitoring has become even more important (and complicated) as more elements are added to the network and the environment evolves, including multiple types of devices, microservices as a key part of application delivery, and, of course, the widespread move to the cloud.

Digital businesses are radically changing how they build and deliver software. Gone are the days of apps that rely solely on in-house tools. Rather, today’s apps are increasingly dependent on external APIs and third-party app providers (which, in turn, are reliant on other APIs and apps). While this type of modularity allows for product flexibility and rapid development, it can be difficult to address any issues that arise. If even one component of this chain breaks, it can have a domino effect on its dependents, whereas a similar failure in historically closed systems would have just led to an isolated incident. As such, if your product relies on external APIs, it's important for you to monitor more than just availability — you’ll also need to keep tabs on performance, data validation and processes, feature changes and security.

In this blog post, we’ll discuss how API monitoring works, key metrics to watch, the benefits and why it’s important to the enterprise as well as how you can get started implementing API monitoring in your organization.

What is an API?

An API is a set of programming instructions and standards for accessing a web-based software application or service. APIs give DevOps professionals instructions about how to interact with services and can be used to connect data among different systems. Entire businesses and applications are built on open APIs, relying on the ability to pass data back and forth between systems.

---

---

APIs may be private, programmer/app-facing APIs used within an internal organization or they may be public, consumer-facing APIs, and have numerous use cases. Private APIs make businesses more agile, flexible and powerful. Public APIs help businesses connect to offer new integrations and build new partnerships.

APIs provide the functionality for different application elements to interact with one another; the points where they connect are known as endpoints.

APIs provide instructions about how to interact with services, and connect data with frontend and backend systems.

How API Monitoring Works

API monitoring can be used to track APIs for availability, functionality, speed and performance issues, and often can be facilitated with automation.

Availability: API monitoring checks to see if API-connected resources are available, working properly and responding to calls. With the increasing interdependence of applications on other apps and services, API monitoring is often extended to monitoring the availability of the resources your APIs rely on as well. API monitoring can alert administrators about any breaks in the chain of dependence and allow IT teams to act appropriately to ensure that their website or app remains online.

Performance: Even if an API returns calls correctly, its application performance may not be up to standard. How quick are the responses? Is the HTTP response code optimized? Are the response times degrading? Does the API’s performance vary in different environments, such as in development versus production?

Any of these performance metrics, whether caused by the APIs you call, backend services, or even other APIs down the line, will affect performance and require some kind of performance monitoring of the site or app. API monitoring will alert you to these degradations with notifications and let you respond.

Data validation and multi-step processes: Your API may be available and responding to requests, but it could send back incorrect data or data formatted in unexpected ways. Monitoring for data validation ensures you’re getting the right responses in the correct format, which is especially important for multi-step processes once you’ve received the initial response. Data validation will let you know if these processes are working as expected.

Monitor for integrations with third party and partner APIs: Apps and services such as Slack and other web applications and web services often rely on both managed and third-party APIs, and frequently users can’t tell the difference between the two. An API monitoring solution can give you visibility into the performance of third-party and partner APIs (such as those of AWS, credit card providers and others) in addition to those you manage. This way, you’ll not only be able to hold partners accountable, but also know who to contact should any issues arise.

Monitor for feature changes: When you have functionality that depends on the performance of an external service, you’ll want to ensure your app remains compatible with the service. Whether the changes are a result of new releases or bug fixes, API monitoring can alert the end user if the base code stops working with an API-driven service following a change or upgrade.

Monitor for security: The more an organization opens its network to outside connections, the higher the opportunity for malicious actors to take advantage of any vulnerabilities. When you use an API, you’re assuming that API is secure and that the security settings are maintained. You’re also assuming the risk of any other APIs that feed into it. API monitoring ensures authentication and ensures the security of APIs connected to your apps by searching for anomalous behavior that could signify a security breach and identify threats in real time.

API monitoring can be used to track APIs for availability, performance, data validation, integrations, feature changes and security.

API Testing and Monitoring Frameworks

An API testing and monitoring framework is the software used to implement API monitoring, which can include tools such as JSON, Pagerduty, oauth and Postman. There are dozens of options available, from open-source packages that allow developers to write custom API monitoring and application monitoring protocols for their own organization, to commercial packages that provide more work upfront for real users. The question of how best to set up an API monitoring framework depends on the individual needs, characteristics and budget of the organization.

Monitoring API Endpoints

APIs provide the functionality for different network elements to interact with one another; the points where they connect are known as endpoints. Therefore, when you are monitoring the best API for security, performance or other factors, you are by default monitoring the endpoints as well. An API endpoint can be functioning properly but not communicating with its intended partner in the network. A comprehensive API monitoring solution will allow you to see when a connection problem occurs at an API endpoint.

The Importance of API Monitoring

APIs have been key in helping businesses and organizations grow, scale and provide new services that users want and need — and ultimately improve end-user experience — without being forced to rewrite their core applications. The rapid rise of e-commerce (including the shifts to SaaS businesses driven by the COVID-19 pandemic) has been made possible in large part by APIs, as companies have been able to add new services and functions, adjust pricing and improve SLAs in order to become online retailers.

APIs have made it possible for companies to provide multiple services on their websites without necessarily creating the applications themselves. Enterprise companies have also been able to increase the number of tools available to their employees by using APIs.

In other words, APIs now touch nearly every aspect of modern applications, networks and infrastructure. Failure to monitor API performance today is tantamount to a failure to follow basic security protocols or track IT performance.

Key API Metrics

When it comes to API monitoring, you want to ensure that an API is functioning properly — doing the job it was designed to do and doing it within the correct performance parameters.

• Availability or uptime: In order for APIs to be effective, they need to be working. Monitoring for uptime is the simplest metric to determine an API's general usability. Uptime is measured as a percentage of total time the API is available. The typical target is 99.9% or even 99.99% uptime. Some users flip the statistics and report on the percentage of downtime.
• Error rate: Error rate measures how often an API does not return the intended result. An error can indicate a problem with the API itself, or with the services the API calls.
• CPU and memory usage: A host server showing an unusually high CPU or memory usage could be a sign that the API is not functioning properly.
• API consumption: API consumption measures how often an API is used and can be calculated as requests-per-minute, requests-per-second, or queries-per-second.
• Response time or latency: Response time (also called latency) reflects how long an API takes to respond when called. There are many factors that can affect API response time, so it is not a reliable metric to use by itself to monitor API performance.

 Key API metrics include availability, error rate, CPU and memory usage, API consumption and latency, to ensure APIs are functioning properly.

Common Benefits of API Monitoring

With APIs controlling and enabling so much of the data and interactions across an organization’s entire network, API monitoring is an essential element for ensuring app and site performance, security, uptime, recovery from outages, user monitoring and nearly every other aspect of an organization’s network performance. 

API monitoring can:

• Track the availability of critical APIs
• Capture and trend an API’s response time
• Ensure API functionality by validating the API’s response values and structure
• Alert on any conditions indicating a broken or poorly performing API, allowing teams to get ahead of issues before they affect users or breach SLAs

Here are a few use cases API monitoring can support:

• Testing complex, multi-step API flows
• Monitoring availability and response time from geographies around the world
• Tracking and enforcing performance SLAs of third-party APIs
• Verifying correctness of API responses
• Testing the entire create, read, update, delete (CRUD) life cycle of a data object via an API
• Handling complex, token-based API authentication systems
• Monitoring application status pages

Common Challenges of API Monitoring

Because APIs are integral to so many aspects of an organization’s entire network, the challenges of API monitoring revolve around scale, experience and comprehensive application.

• The "Black Box": Sometimes applications and microservices are created in such a way as to protect proprietary information. They may not have an API, or their API may not expose enough data or features to provide a full picture of their performance
• Siloed data: Because APIs connect multiple objects across a network, both inside and outside, they can encounter multiple types of data in different formats. If an API is trying to connect two different devices that share data in different formats, the disparate formatting will have to be resolved before the API can perform its function.
• System performance: Some API monitoring solutions can degrade system performance by affecting the ability of the API to do its job. When selecting an API monitoring solution, it's important to find one that doesn't slow things down.

Getting Started with API Monitoring

API monitoring is generally provided by a software and hardware vendor who specializes in the field. The most effective way to get started with API monitoring is to research API monitoring vendors and follow your organization’s request for proposal (RFP) process. Here are some considerations to keep in mind as you begin the process.

When starting to monitor APIs, consider both:

• APIs that your website or native application rely on for critical data or processes
• APIs that you manage that customers, end users, or developers rely on for data or processes

When monitoring both of these types of APIs, it’s important to test:

• Availability: Is this API endpoint up? Is it returning an error?
• Response time: How quickly is the API returning responses? Is the response time degrading over time? Is the response time worse in production than in pre-production?
• Data validation: Is the API returning the correct data in the right format?
• Multi-step processes: Can I successfully save and re-use a variable from this API? Does authentication work as expected? Can I complete a transaction with data from this API?

---

---

The Bottom Line: API monitoring is critical in an API-connected world

Companies increasingly rely on APIs to perform business-critical functions. And yet, many either aren’t monitoring their APIs, or they’re failing to implement best practices. Because of the high stakes associated with any type of downtime, it’s essential for companies to monitor dependencies to ensure that everything is working as expected. Businesses might not have any control over the third-party tools they rely on, but with comprehensive monitoring, they can know right away if they need to take action to keep their product up and running.

What is Splunk?

This posting does not necessarily represent Splunk's position, strategies or opinion.