Distributed Throttling

WSO2 API Microgateway has an in-memory mechanism by default, to handle throttling (node-level throttling).  In a deployment with multiple microgateways, throttling becomes a challenge with node local throttling as the throttling decision is made based on the local counter within each node. If we proceed with the node local throttling in such environment, the API user would be allowed to send multiples of the throttling limit.I.e. if the throttling limit is set to 10, if we have 3 gateways in a cluster, it will allow 30 requests to pass to the backend before all three gateways throttle out requests. This will put an unexpected load on the backend. To address this requirement, the API Microgateway supports distributed throttling where it is able to work with a central traffic management solution. In this case, multiple microgateways can connect with WSO2 API Manager (WSO2 Traffic Manager) and perform rate-limiting precisely.

Note

If you start the WSO2 API Manager without providing any profile, it runs as All in One Node (All the profiles are activated). For testing purposes, you can simply start the API Manager following the quick start guide and test.

Enabling distributed throttling

  1. Let's create a microgateway project

    Create a project using the command given below.

    micro-gw init <project_name>
    micro-gw init petstore
    Project 'petstore' is initialized successfully.
  2. Add the API definition(s) to petstore/api_definitions directory. A sample open API definition can be found here. Then provide the API throttling tier using the following extension at the API level.

    x-wso2-throttling-tier : "5PerMin"    
  3. Create and deploy the throttling policy in the Traffic Manager. (For this example you should deploy "5PerMin" policy in Traffic Manager) The relevant documentation can be found here.

  4. Build the microgateway distribution for the project using the following command:

    micro-gw build <project_name>
    micro-gw build petstore
    BUILD SUCCESSFUL
    Target: /Users/praminda/Documents/workspace/mgw/320/rc2/petstore/target/petstore.jar

    Once the above command is executed, An executable file (petstore/target/petstore.jar) is created to expose the API via WSO2 API Microgateway

  5. Open <MGW-RUNTIME-HOME>/conf/micro-gw.conf file.

  6. Enable the enabledGlobalTMEventPublishing property found inside the throttlingConfig tag. This will allow the API Microgateway to connect with the central traffic manager.

    [throttlingConfig]
      enabledGlobalTMEventPublishing=true
  7. In the "micro-gw.conf" file under [throttlingConfig]  configure the following. The purpose here is to configure the message broker.

    Property Default Value Description
    jmsConnectionProviderUrl amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672' The message broker connection URL of WSO2 API/Traffic Manager
    jmsConnectionUsername "" The username used to establish the message broker connection
    jmsConnectionPassword "" The password used to establish the message broker connection

    The message broker connection URL. For e.g. a WSO2 API instance can be used as the Traffic Manager. In such an instance, the URL will point to the message broker inside the API Traffic Manager instance. In the micro-gw.conf file under [throttlingConfig.binary], we should list down all the configurations related to event publishing.

    Property Default value Description
    enabled true Enable the binary event publisher. If it is false, http event publisher will be enabled
    username "admin" The user name used for authentication prior to publishing events via binary publisher
    password "admin" The password used for authentication prior to publishing events via binary publisher

    The binary receiver URL(s) needs to be added as an Array using the key [[throttlingConfig.binary.URLGroup]]. If multiple receivers are provided, the microgateway will publish events to all of them in parallel.

    Property Default value Description
    receiverURL "tcp://localhost:9611" The URL to which the thorttle events are sent.
    authURL "ssl://localhost:9711" The URL to which the credentials required for authentication, are sent.

  8. If you are using API Manager 3.2.0 or later version, you need to configure the API Manager Eventhub. This step is not required if you need to do only the resource level throttling or API level throttling.

  9. Finally, the added configurations in micro-gw.conf would look like this.

    [throttlingConfig]
      enabledGlobalTMEventPublishing=true
      jmsConnectionProviderUrl = "amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'"
    
      [throttlingConfig.binary]
        enabled = true
        username = "admin"
        password = "admin"
        [[throttlingConfig.binary.URLGroup]]
          receiverURL = "tcp://localhost:9611"
          authURL = "ssl://localhost:9711"
    
    [apim.eventHub]
      enable = true
      serviceUrl = "https://localhost:9443"
      internalDataContext="/internal/data/v1/"
      username="admin"
      password="admin"
      eventListeningEndpoints = "amqp://admin:admin@carbon/carbon?brokerlist='tcp://localhost:5672'"
  10. Execute the following command to start WSO2 API Microgateway with new configurations.

    gateway <path-to-executable-file>
    gateway /Users/kim/petstore/target/petstore.jar

Conditional throttling

There can be situations where certain APIs require more granular level of throttling. Assume you want to provide limited access to a certain IP range or a type of client application (identified by User-Agent header). For these scenarios, a simple throttle policy with API/resource level limits is not sufficient. To address complex throttling requirements as above, microgateway is capable of throttling requests based on several conditions. The following types of conditions are supported.

  1. Specific IP or IP range conditions.
    This condition can be used to provide specific limits to a certain IP address or a range of IP addresses.

  2. Header conditions.
    This condition can be used to set specific limits to a certain header value.

  3. Query parameter conditions.
    Same as the header conditions, this allows applying a specific limit to a certain query parameter value.

  4. JWT claim conditions.
    This type of condition will evaluate the backend jwt and check if it has a specific claim value in it to set the throttle limit.

Configure and enable conditional throttling

  1. Open micro-gw.conf file in <MGW_HOME>/conf directory.
  2. Add/Enable below configurations to enable the required condition type.

    [throttlingConfig]
      enabledGlobalTMEventPublishing = true
      enableHeaderConditions = true
      enableQueryParamConditions = true
      enableJwtClaimConditions = true
  3. Define the Advance Throttle Policy containing the required conditions in WSO2 API Manager. To do this follow Adding a new advanced throttling policy

  4. Conditional throttle policies are an advanced set of API and Resource level policies. Therefore you need to define the required policy to apply in the OpenAPI definition. To do that, define x-wso2-throttling-tier extension with the Advance Throttle Policy name you defined in API Manager in the above step. This extension can be defined in both API and Resource levels.

    x-wso2-basePath: /petstore/v1
    x-wso2-throttling-tier: IPPolicy
    x-wso2-production-endpoints:
      urls:
        - https://petstore.swagger.io/v2
    paths:
      "/pet/findByStatus":
          get:
            summary: Finds Pets by status
            x-wso2-throttling-tier: 10kPerMin
Top