Geospatial Portal

Enforcing Per-Tenant WMS Request Quotas with Nginx

This guide configures a per-tenant request budget for WMS traffic at the Nginx edge, so that one agency’s mapping application cannot consume the rendering capacity that every other tenant is relying on. It belongs to the API Gateway, Rate Limiting & Quota Enforcement topic, within the wider Core Portal Architecture & Security Boundaries framework.

Prerequisites

  • Nginx 1.18 or newer with ngx_http_limit_req_module and ngx_http_limit_conn_module (both are in the standard build).
  • A working authentication step at the edge that resolves a request to a tenant — an auth_request subrequest or a JWT module that populates a variable. The token side is covered in mapping OIDC group claims to GeoNode roles.
  • A tenant list with an agreed request rate per tenant, and a default for tenants with no specific agreement.
  • Shell access to reload Nginx, and a staging vhost to test against before touching production.

The shape of the budget

A tenant budget is not one number. Nginx offers two independent controls and they bound different things: limit_req bounds the arrival rate of requests, and limit_conn bounds the number of simultaneous requests in flight. A tenant issuing sixty tile requests per second, each completing in milliseconds, is well behaved; a tenant holding sixty concurrent uncached renders is occupying the whole renderer regardless of how slowly it started them.

Rate limits and concurrency limits bound different behaviours Two columns, request rate and concurrency, each with what it stops and what it still permits, showing why a tenant budget needs both. limit_req — arrival rate stops a crawler walking the tile pyramid still permits a few very slow renders held open, occupying workers indefinitely measured over a short window limit_conn — in flight at once stops a tenant occupying the renderer still permits an unlimited series of fast cheap requests, one after another measured instantaneously

Set both, and size them from different observations: the rate from a busy tenant’s normal peak with headroom, and the concurrency from how many simultaneous renders the backend can serve divided by the number of tenants you are willing to have compete at once.

Step-by-step implementation

1. Resolve the request to a tenant

The limit must key on something the tenant cannot forge. Take the tenant from a verified token claim, populated by the authentication subrequest, and fall back to the source address only for unauthenticated traffic.

# /etc/nginx/conf.d/00-tenant.conf  (http context)

# The auth subrequest returns X-Tenant-Id on success; auth_request_set lifts it
# into a variable this vhost can use as a limit key.
map $upstream_http_x_tenant_id $wms_tenant {
    default   $upstream_http_x_tenant_id;
    ""        "anon:$binary_remote_addr";   # unauthenticated: fall back to address
}

# Per-tenant zones. 10m holds roughly 160k distinct keys — far more than the
# tenant count, and the headroom absorbs the anonymous fallback keys.
limit_req_zone  $wms_tenant zone=wms_rate:10m rate=30r/s;
limit_conn_zone $wms_tenant zone=wms_conn:10m;

limit_req_status  429;
limit_conn_status 429;

2. Give named tenants their own rate

Nginx zones carry a single rate, so per-tenant differentiation is expressed by routing tenants into different zones rather than by parameterising one. Two or three tiers cover nearly every portal.

# Tier each tenant. Anything unlisted lands on the standard tier.
map $wms_tenant $wms_tier {
    default              standard;
    "~^anon:"            public;
    "tenant-highways"    large;      # bulk consumer, contractual allowance
    "tenant-planning"    large;
    "tenant-parish-24"   small;
}

limit_req_zone  $wms_tenant zone=wms_public:10m   rate=5r/s;
limit_req_zone  $wms_tenant zone=wms_small:10m    rate=10r/s;
limit_req_zone  $wms_tenant zone=wms_standard:10m rate=30r/s;
limit_req_zone  $wms_tenant zone=wms_large:10m    rate=120r/s;

3. Apply the limits on the WMS location

Each tier gets its own location, selected by an internal redirect, so that only the matching zone’s limit is evaluated. Burst is sized to a viewport of tiles — a slippy map opening thirty requests at once is normal use, not abuse — and nodelay admits that burst immediately instead of smearing it across a second.

server {
    listen 443 ssl;
    server_name portal.example.gov;

    location /geoserver/wms {
        auth_request      /_authz;
        auth_request_set  $upstream_http_x_tenant_id $upstream_http_x_tenant_id;

        # Route to the tier-specific location without a client-visible redirect.
        try_files /dev/null @wms_$wms_tier;
    }

    location @wms_public {
        limit_req  zone=wms_public burst=20 nodelay;
        limit_conn wms_conn 4;
        proxy_pass http://geoserver_backend;
    }

    location @wms_standard {
        limit_req  zone=wms_standard burst=60 nodelay;
        limit_conn wms_conn 12;
        proxy_pass http://geoserver_backend;
    }

    location @wms_large {
        limit_req  zone=wms_large burst=200 nodelay;
        limit_conn wms_conn 40;
        proxy_pass http://geoserver_backend;
    }
}

4. Tell the caller what happened

A bare 429 with no guidance produces an immediate retry and a support ticket. Emit Retry-After on every rejection, and return a short, machine-readable body so a client can distinguish throttling from an outage.

    # Inside the server block
    error_page 429 = @throttled;

    location @throttled {
        internal;
        default_type application/json;
        add_header Retry-After 5 always;
        add_header Cache-Control "no-store" always;
        return 429 '{"error":"rate_limited","retry_after":5,"docs":"https://www.geospatialportal.org/core-portal-architecture-security-boundaries/api-gateway-rate-limiting-and-quota-enforcement/"}';
    }

5. Log the rejection with its tenant

A rejection you cannot attribute is a rejection you cannot act on. Add the tenant and the limit status to the access log format so the weekly review is a query rather than an investigation.

log_format ogc_limits '$remote_addr $wms_tenant $wms_tier $status '
                      '$request_time "$request" '
                      'limit_req=$limit_req_status limit_conn=$limit_conn_status';

access_log /var/log/nginx/ogc_access.log ogc_limits;

Roll it out in observation mode first

Turning limits on for real traffic without knowing who they would have hit is the reliable way to break a partner integration on a Monday morning. Nginx has no native dry-run, but the same effect is available by setting the limits deliberately high and reading the $limit_req_status field, which reports DELAYED or REJECTED even when the configured rate is never reached in practice.

Three stages between configuring a limit and enforcing it Observation, single-tenant pilot, and full enforcement, each with what is changed and the condition for moving on. 1 · observe limits set far above observed peaks exit: you know each tenant's real peak over a full week 2 · one tenant real limits, applied to a tenant who agreed exit: a week with no unexplained rejections 3 · everyone all tiers enforced, with the tenants notified then: review rejections weekly, not only on alert Skipping stage one produces the classic outcome: the first enforcement window is also the first time anyone learns what a tenant's normal traffic looks like, and the limit is reverted under pressure the same day.

Sizing the numbers from real traffic

The limits above are only as good as the values in them, and the values come from measurement rather than from judgement. Three figures are worth extracting from a week of access logs before enforcing anything.

The first is each tenant’s peak request rate over a one-second window, not an average — averages hide the burst that a rate limit will actually reject. The second is the concurrency each tenant reaches, which the log gives indirectly as the number of overlapping requests; it is usually far lower than teams expect for interactive use and far higher for anything scripted. The third is the distribution of request cost, approximated by response time, because a tenant whose requests are uniformly fast can be given a generous rate safely while one issuing slow uncached renders cannot.

Three measurements, three limits Peak rate, peak concurrency and cost distribution, each with the limit it sizes and how much headroom to leave. MEASURE SIZES HEADROOM peak rate, 1s window not the average the rate directive about twice the observed peak peak overlap requests in flight at once the connection limit the backend ceiling, divided fairly response-time spread which tier a tenant belongs in slow callers get a tighter rate

Round the resulting numbers generously. A limit that a well-behaved tenant meets occasionally will be reported as a fault, investigated, and eventually removed; one that only a pathological caller meets keeps its credibility and survives.

Verification 

# 1. The configuration parses before any reload touches production
nginx -t
#   expect: syntax is ok / test is successful

# 2. A normal viewport burst is admitted in full (no 429 among 30 rapid tiles)
seq 0 29 | xargs -P 30 -I{} curl -s -o /dev/null -w '%{http_code}\n' \
  -H "Authorization: Bearer $TENANT_TOKEN" \
  "https://portal.example.gov/geoserver/wms?service=WMS&request=GetMap&layers=base&bbox=0,0,1,1&width=256&height=256&format=image/png" \
  | sort | uniq -c
#   expect: 30 responses of 200

# 3. Sustained excess is rejected, and the rejection carries Retry-After
for i in $(seq 1 400); do
  curl -s -o /dev/null -D - -H "Authorization: Bearer $TENANT_TOKEN" \
    "https://portal.example.gov/geoserver/wms?service=WMS&request=GetMap&layers=base&bbox=0,0,1,1&width=256&height=256&format=image/png"
done | grep -E '^HTTP/|^Retry-After' | sort | uniq -c
#   expect: a mixture of 200 and 429, and a Retry-After on every 429

# 4. One tenant's excess does not affect another tenant
curl -s -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $OTHER_TENANT_TOKEN" \
  "https://portal.example.gov/geoserver/wms?service=WMS&request=GetCapabilities"
#   expect: 200, while the tenant above is still being throttled

# 5. Rejections are attributable in the log
awk '$4 == 429 {print $2}' /var/log/nginx/ogc_access.log | sort | uniq -c | sort -rn | head
#   expect: rejections concentrated on the tenant you were testing

Check 4 is the one that proves the configuration is doing what it claims. If the second tenant is also throttled, the limit is keyed on something shared — usually because the tenant variable was empty and every request fell back to the same anonymous key.

Troubleshooting matrix

Symptom Likely cause Fix
Every request is throttled together regardless of tenant $wms_tenant resolves empty, so all requests share one key Confirm auth_request_set runs before the limit; log the variable
Interactive maps stutter but scripts are fine Burst too small, or nodelay omitted on the tile tier Size burst to a full viewport and add nodelay
429s appear with no corresponding entry in the tenant log Rejection happened before the auth subrequest set the tenant Move the address-based backstop after authentication, or log both keys
A tenant exceeds its rate with no rejections Its requests match a different location with no limit Confirm the try_files redirect covers every OGC path, not only /wms
Nginx reports “zone already declared” on reload The same zone name defined in two included files Keep all limit_req_zone directives in one file in the http context
Concurrency limit rejects during large exports One legitimate slow request per connection, several in parallel Raise limit_conn for that tier, or route exports to a separate path
Memory use climbs steadily on the edge Anonymous fallback creating a key per source address Shorten the zone or key anonymous traffic on a coarser network prefix

FAQ

Why not use one zone with per-tenant rates from a map?

Nginx fixes the rate when the zone is declared, so a single zone cannot carry different rates for different keys. Routing tenants into tier-specific zones is the supported way to express that, and it has a useful side effect: the tier becomes an explicit, reviewable list rather than an emergent property of a large map.

How large should the shared memory zone be?

Ten megabytes holds on the order of 160,000 keys, which is far more than any realistic tenant count. The reason to size it generously is the anonymous fallback, which creates a key per source address; if that zone fills, Nginx evicts the oldest entries and limits become unreliable rather than failing loudly.

Does nodelay defeat the purpose of the limit?

No — it changes when excess is rejected, not whether. Without it, requests inside the burst are queued and released at the configured rate, which makes an interactive map fill in slowly. With it, the burst is served immediately and anything beyond the burst is rejected at once. The sustained rate is enforced identically in both cases.

Should the limits apply to health checks and monitoring probes?

Exclude them, by keying probes to a dedicated identity and giving that identity its own generous allowance. Probes are deliberately regular and would otherwise consume a share of a real tenant’s budget, and — more importantly — a throttled probe reports a false outage.

What is the right response when a tenant asks for a higher limit?

Ask what they are doing, not how much they want. Most requests for a higher limit turn out to be a client that could tile, page, or cache instead, in which case raising the limit hides an inefficiency that will return at a larger scale. Where the need is genuine, raise it with a recorded expiry and review it at that date.

Up one level: API Gateway, Rate Limiting & Quota Enforcement.