WSO2 API Manager 4.1.0 API Throttling Model Internals

API consumers can be very different from large enterprise system establishing a B2B interaction to a developer testing an API with “Postman” tool.

Users may be behind this interaction but will always work with the interface that abstracts the actual API communication. Therefore always there is the domain object : “application” which makes the API call.

The concept of “application” is modelled as a business object in WSO2 API Manager design. It is composed of multiple attributes. Some of these attributes can be externally managed and others are auto generated by the runtime. Please see below.

Image 1.0 : The “Application” domain object

Developer can use use Dev Portal REST APIs or Developer Portal UI to create an “application” as shown below.

Image 2.0 : Application API and Developer Portal UI side by side comparison

A corresponding entry is made on the “AM_APPLICATION” table under WSO2AM_DB schema as shown below.

Image 3.0 : AM_APPLICATION table from H2 database

WSO2 API Manager uses the concept of “Application” to decouple the consumer from APIs. It allows generate and use a single key or a token for multiple APIs and subscribe multiple times to a single API with different SLAs. The relationship is shown below.

Image 4.0 : How API token relates to application and subscription.

An API token is issued per an “Application”. An API consumer can use the same token while the API subscriptions being changed. When an API subscription is made an entry goes in to “AM_SUBSCRIPTION” table. Following diagram shows the relationship between SUBSCRIPTION_ID, API_ID and APPLICATION_ID from the “AM_SUBSCRIPTION” table.

Image 5.0 : AM_SUBSCRIPTION table from H2 database

The throttling policies are created independent of APIs and Applications. Again this can be done through the UI or Developer Portal REST APIs. They are maintained in a throttling policy registry. There are three types of policies.

1. Application policies
2. Subscription policies
3. Advanced policies
4. Custom policies

Custom and advanced throttling policies allow you to build sophisticated API request control mechanisms. Advanced policies can be applied at individual API or API operation level. Custom policies are slightly different and applied global.

You can access these policies using the WSO2 API Manager “Admin” Portal as shown below.

Image 6.0 : Custom Rate Limiting page from the Admin Portal

Advanced throttling policy page from WSO2 API Manager “Admin” Portal as shown below with an explanation of how it works.

The DSL used to define throttling policy is “Siddhi streaming SQL”. It uses a in-memory event stream to capture events which represents the injected API request meta data. Then it performs a “SQL” like query against the event stream to pick only the relevant events and perform a logic (throttling decision) to understand whether throttling is required. The implementation model is shown below.

A script file with an extension “.siddhiql” is created in the folder path : <APIM_HOME>/reposiory/deployment/server/executionplans for each throttling policy as shown below.

You can use WSO2 Carbon console to edit and validate Siddhi SQL queries coming from all types of throttling policies.

How policies are linked to the API request flow is shown below.

These policies can be linked at different stages of the request flow as shown below.

WSO2 API Manager uses an “Event Hub” to communicate throttling decisions. The API gateway sends request event through a throttling handler to the throttling engine to make decisions as shown below.

Once a decision is made to throttle out the request, decision is informed to the gateway as shown below.