-
Notifications
You must be signed in to change notification settings - Fork 6
Home
Guardian originally started as a global rate limiting system with an asynchronous counting mechanism which traded exact limit enforcement for low latency. It had a single, configurable rate limit that was applied to every request from a given IP using a fixed window counting technique.
Guardian still maintains this behavior, but has evolved into a more flexible rate limiting system with some new capabilities that make it more effective at protecting services.
This document will illustrate the core features of Guardian, so that operators of Guardian can confidently make decisions when changing configurations.
A Limit is the mechanism for configuring the conditions when rate limiters in Guardian should block requests from clients
Limits are used to configure various features in Guardian.
They are dynamically configurable in that Guardian does not need to restart in order to receive new limits. This allows operators to adjust these limits on the fly, as conditions may change in upstream services.
Count - The number of requests that will be permitted in the specified time period. Once the count is reached, rate limiters will block requests for the remainder of the time period.
Duration - The time period in which the counts will be accumulated before being reset.
Enabled - Flag to enable/disable this limit. A disabled limit will prevent Guardian from blocking requests.
Note that a longer duration implies that a client IP can be blocked for a longer period of time if they were to surpass the count at the beginning of the time window.
Guardian is configured using YAML configuration files to customize its behavior. These fields in these files specify metadata and a specification for each configuration. If you are upgrading from a previous version of Guardian configured using command-line arguments, see Updating Guardian to use YAML configuration files.
Configuration files are applied via guardian-cli to a running Guardian instance. Each configuration file contains these fields:
version: The version of the config file format. This allows the format to be changed or extended without breaking existing deployments. These versions are specific to the config file’s kind (see below), so different kinds of configuration files can potentially be at different version numbers.
kind: The kind of configuration that the file represents. This can be one of: GlobalRateLimit, GlobalSettings, RateLimit, or Jail.
name: A name to identify the configuration, which must be unique with respect to the config file’s kind. This name is also used to identify a RateLimit or Jail to delete.
description: A human-readable description of the config file.
spec: The specification of the configuration file. The specification for each kind of configuration is different (see below for example configuration files for each kind).
Multiple configurations, not necessarily of the same kind, can be applied by including multiple YAML documents in one file, separated with the YAML document start indicator ---.
Generally:
guardian-cli --redis-address=<redis_address> apply <config_file_path>
Example:
guardian-cli --redis-address=localhost:6379 apply globalratelimit.yml
Generally:
guardian-cli --redis-address=<redis_address> get <kind>
Example:
guardian-cli --redis-address=localhost:6379 get RateLimit
Generally:
guardian-cli guardian-cli --redis-address=<redis_address> delete <kind> <name>
Example:
guardian-cli --redis-address=localhost:6379 delete RateLimit LoginPage
The global rate limit is the default rate limit that applies to all requests that are not captured by other configured rate limiters.
It is intended to be the default rate limiting enforcement preventing excessive use of any upstream service.
Due to the nature of this feature processing all requests not matched by other rate limiters, it is important to set a conservative value for this rate limit. Otherwise, you risk rate limiting legitimate client traffic.
Generally, apply a YAML configuration file with the following format (see Configuration files):
version: "v0"
kind: GlobalRateLimit
name: <name>
description: <description>
globalRateLimitSpec:
limit:
count: <limit_count>
duration: <limit_duration>
enabled: <true|false>Example:
version: "v0"
kind: GlobalRateLimit
name: GlobalRateLimit
description: GlobalRateLimit
globalRateLimitSpec:
limit:
count: 5
duration: 1m
enabled: trueGlobal settings are settings that apply to an entire Guardian deployment. Currently, the only global setting configures whether Guardian should actually limit requests or simply report without applying any limiting.
It may be necessary to disable rate limiting globally if too many valid requests or the wrong requests are being limited. Setting reportOnly to true will prevent all forms of rate limiting (global, conditional, or jail) from being applied.
Generally, apply a YAML configuration file with the following format (see Configuration files):
version: "v0"
kind: GlobalSettings
name: <name>
description: <description>
globalSettingsSpec:
reportOnly: <true|false>Example:
version: "v0"
kind: GlobalSettings
name: GlobalSettings
description: GlobalSettings
globalSettingsSpec:
reportOnly: falseThe Whitelist is a list of CIDR blocks that cannot be blocked by Guardian. The whitelist is the first part of the processing pipeline for Guardian. If a client IP address is in a CIDR range in the whitelist, Guardian will immediately let the request through.
This allows for trusted CDNs, synthetic testing, and webhooks to reach upstream services without being interrupted.
Generally:
guardian-cli --redis-addres=<redis_address> add-whitelist <cidr_range>
Example:
guardian-cli add-whitelist 1.1.1.1/32 --redis-address=localhost:6379
The Blacklist is a list of CIDR blocks that are always blocked by Guardian. The blacklist is processed immediately after the whitelist so that we don’t waste resources attempting to keep counts of these malicious requests. If a client IP address is in a CIDR range in the blacklist, Guardian will immediately block the request.
It is intended to efficiently block malicious users or clients that have no reason to contact an upstream service.
Due to the nature of rate limiting, malicious users could basically hit the rate limit, wait for the duration period to end, and then begin to send traffic immediately after. The blacklist prevents this behavior, but requires manual intervention to add or remove CIDR ranges from the list.
Generally:
guardian-cli --redis-addres=<redis_address> add-blacklist <cidr_range>
Example:
guardian-cli add-blacklist 1.1.1.1/32 --redis-address=localhost:6379
Rate Limits are rate limits that are only applied given certain conditions. Currently, the only condition Guardian uses is the request path (route rate limiting).
Since the global rate limit has to be fairly conservative, standard rate limits allow Guardian to enforce strict limits based off conditions of the request.
How to Configure Generally, apply a YAML configuration file with the following format (see Configuration files:
version: "v0"
kind: RateLimit
name: <name>
description: <description>
rateLimitSpec:
limit:
count: <limit_count>
duration: <limit_duration>
enabled: <true|false>
conditions:
path: <request_path>Example:
version: "v0"
kind: RateLimit
name: "/foo/bar"
description: "/foo/bar"
rateLimitSpec:
limit:
count: 10
duration: 1m
enabled: true
conditions:
path: "/foo/bar"Jail is a namesake of a similar concept from fail2ban.
https://docs.plesk.com/en-US/obsidian/administrator-guide/73382/
In the context of a Guardian, a jail is composed of a Limit and a BanDuration.
Jail - rule that informs Guardian to ban client IP addresses that surpass a configured limit. They are banned for the entire BanDuration from making any requests.
Prisoner - a client IP that is currently banned due to any of the configured Jails. Also referred to as a client IP that has been jailed.
Jailer - uses the jails and a counter to keep track of which Client IPs should be prisoners.
PrisonerStore - used by the Jailer to get the current prisoners and also add prisoners as needed
While we can use Global Rate Limits and Route Rate Limits to help defend against possible abuse of upstream services, it’s still possible for an adversary to hover just below the rate limits for long periods of time which may be unacceptable in certain cases.
With Jails, we can have a powerful tool for automatically banning IPs from using any upstream services for an extended period of time.
Generally, apply a YAML configuration file with the following format (see Configuration files):
version: "v0"
kind: Jail
name: <name>
description: <description>
jailSpec:
limit:
count: <limit_count>
duration: <limit_duration>
enabled: <true|false>
conditions:
path: <request_path>
ban_duration: <duration>Example:
version: "v0"
kind: Jail
name: "/foo/bar"
description: "/foo/bar"
jailSpec:
limit:
count: 10
duration: 10s
enabled: true
conditions:
path: "/foo/bar"
ban_duration: 30s