WSO2 API Microgateway has an in-memory mechanism by default, to handle throttling (node-level throttling). In a deployment with multiple microgateways, throttling becomes a challenge with node local throttling as the throttling decision is made based on the local counter within each node. If we proceed with the node local throttling in such environment, the API user would be allowed to send multiples of the throttling limit.I.e. if the throttling limit is set to 10, if we have 3 gateways in a cluster, it will allow 30 requests to pass to the backend before all three gateways throttle out requests. This will put an unexpected load on the backend. To address this requirement, the API Microgateway supports distributed throttling where it is able to work with a central traffic management solution. In this case, multiple microgateways can connect with WSO2 API Manager (WSO2 Traffic Manager) and perform rate-limiting precisely.
If you start the WSO2 API Manager without providing any profile, it runs as All in One Node (All the profiles are activated). For testing purposes, you can simply start the API Manager following the quick start guide and test.
Enabling distributed throttling¶
Let's create a microgateway project
Create a project using the command given below.
micro-gw init <project_name>
micro-gw init petstore
Project 'petstore' is initialized successfully.
Add the API definition(s) to
petstore/api_definitionsdirectory. A sample open API definition can be found here. Then provide the API throttling tier using the following extension at the API level.
x-wso2-throttling-tier : "5PerMin"
Create and deploy the throttling policy in the Traffic Manager. (For this example you should deploy "5PerMin" policy in Traffic Manager) The relevant documentation can be found here.
Build the microgateway distribution for the project using the following command:
micro-gw build <project_name>
micro-gw build petstore
BUILD SUCCESSFUL Target: /Users/praminda/Documents/workspace/mgw/320/rc2/petstore/target/petstore.jar
Once the above command is executed, An executable file (
petstore/target/petstore.jar) is created to expose the API via WSO2 API Microgateway
enabledGlobalTMEventPublishingproperty found inside the
throttlingConfigtag. This will allow the API Microgateway to connect with the central traffic manager.
In the "micro-gw.conf" file under [throttlingConfig] configure the following. The purpose here is to configure the message broker.
Property Default Value Description
The message broker connection URL of WSO2 API/Traffic Manager
The username used to establish the message broker connection
The password used to establish the message broker connection
The message broker connection URL. For e.g. a WSO2 API instance can be used as the Traffic Manager. In such an instance, the URL will point to the message broker inside the API Traffic Manager instance. In the
[throttlingConfig.binary], we should list down all the configurations related to event publishing.
Property Default value Description
Enable the binary event publisher. If it is
false, http event publisher will be enabled
The user name used for authentication prior to publishing events via binary publisher
The password used for authentication prior to publishing events via binary publisher
The binary receiver URL(s) needs to be added as an Array using the key
[[throttlingConfig.binary.URLGroup]]. If multiple receivers are provided, the microgateway will publish events to all of them in parallel.
Property Default value Description
The URL to which the thorttle events are sent.
The URL to which the credentials required for authentication, are sent.
If you are using API Manager 3.2.0 or later version, you need to configure the API Manager Eventhub. This step is not required if you need to do only the resource level throttling or API level throttling.
Finally, the added configurations in
micro-gw.confwould look like this.
[throttlingConfig] enabledGlobalTMEventPublishing=true jmsConnectionProviderUrl = "amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'" [throttlingConfig.binary] enabled = true username = "admin" password = "admin" [[throttlingConfig.binary.URLGroup]] receiverURL = "tcp://localhost:9611" authURL = "ssl://localhost:9711" [apim.eventHub] enable = true serviceUrl = "https://localhost:9443" internalDataContext="/internal/data/v1/" username="admin" password="admin" eventListeningEndpoints = "amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'"
Execute the following command to start WSO2 API Microgateway with new configurations.
There can be situations where certain APIs require more granular level of throttling. Assume you want to provide limited access to a certain IP range or a type of client application (identified by User-Agent header). For these scenarios, a simple throttle policy with API/resource level limits is not sufficient. To address complex throttling requirements as above, microgateway is capable of throttling requests based on several conditions. The following types of conditions are supported.
Specific IP or IP range conditions.
This condition can be used to provide specific limits to a certain IP address or a range of IP addresses.
This condition can be used to set specific limits to a certain header value.
Query parameter conditions.
Same as the header conditions, this allows applying a specific limit to a certain query parameter value.
JWT claim conditions.
This type of condition will evaluate the backend jwt and check if it has a specific claim value in it to set the throttle limit.
Configure and enable conditional throttling¶
Add/Enable below configurations to enable the required condition type.
[throttlingConfig] enabledGlobalTMEventPublishing = true enableHeaderConditions = true enableQueryParamConditions = true enableJwtClaimConditions = true
Define the Advance Throttle Policy containing the required conditions in WSO2 API Manager. To do this follow Adding a new advanced throttling policy
Conditional throttle policies are an advanced set of API and Resource level policies. Therefore you need to define the required policy to apply in the OpenAPI definition. To do that, define
x-wso2-throttling-tierextension with the Advance Throttle Policy name you defined in API Manager in the above step. This extension can be defined in both API and Resource levels.
x-wso2-basePath: /petstore/v1 x-wso2-throttling-tier: IPPolicy x-wso2-production-endpoints: urls: - https://petstore.swagger.io/v2
paths: "/pet/findByStatus": get: summary: Finds Pets by status x-wso2-throttling-tier: 10kPerMin