Geospatial Portal

Benchmarking Tile Latency with k6

A repeatable procedure for load-testing a map tile endpoint with k6, driving randomized z/x/y requests inside a bounding extent and reporting p95/p99 latency for cache-cold and cache-warm runs.

This is a hands-on companion to MapProxy vs TileServer GL for Tile Delivery and sits within the wider Infrastructure Orchestration & Configuration Management practice; read the parent guide first for why the two servers have different CPU and bandwidth profiles. Here the focus is narrow: how to measure real tail latency under load so a scaling threshold comes from data, not a vendor claim.

Prerequisites

  • k6 0.49+ installed (k6 version) — the Grafana load generator; the summary-export API used below is stable from 0.46 onward.
  • A reachable tile endpoint — either a MapProxy WMTS/TMS URL or a TileServer GL /data/<id>/{z}/{x}/{y}.pbf URL. Have both if you intend to compare them.
  • The z/x/y URL template and a bounding extent (min/max longitude and latitude) of a region that actually contains tiles, so requests are not all 404 misses.
  • Enough client bandwidth and CPU on the load-generator host that k6 itself is not the bottleneck; run it off the server under test.
  • Optional: a way to flush the tile cache between runs (restart MapProxy’s cache backend, or point at a freshly seeded archive) so cache-cold numbers are honest.

The harness below shows how the script turns a bounding extent into randomized tile requests, checks each response, and lets the thresholds decide the pass/fail exit code.

k6 tile-latency harness from staged virtual users to a threshold pass or fail Staged virtual users ramp from zero to a plateau and back down. Each virtual-user iteration picks a random zoom then a random x and y column and row inside the configured bounding extent, builds the tile URL, and sends an HTTP GET to the tile server under test. The response feeds a check for status 200 and a non-empty body, and its duration is recorded into the http_req_duration metric. When the run ends, the p95 and p99 thresholds plus the failure-rate threshold evaluate the collected metrics and set the process exit code to pass or fail. A summary JSON is exported alongside. stages (ramp VUs) 0 → plateau → 0 Random z/x/yinside bbox extent HTTP GET tileserver under test check() 200non-empty body http_req_durationrecorded per request request thresholds decide exit p95 < 400 ms p99 < 900 ms failure rate < 1% pass / fail summary.json exported alongside

Step-by-step implementation

1. Parameterize the endpoint and extent

Keep the base URL and tile-template out of the script body so the same file can hit MapProxy or TileServer GL without edits. k6 reads environment variables through __ENV. Decide your extent as a bounding box in longitude/latitude; the script converts lon/lat to tile columns and rows so requests land on real data.

# Compare the two servers by only swapping BASE_URL / TILE_PATH.
# MapProxy (TMS-style):
export BASE_URL="https://tiles.example.org"
export TILE_PATH="/tms/1.0.0/parcels/webmercator/{z}/{x}/{y}.png"

# TileServer GL (vector tiles) — same script, different env:
# export BASE_URL="https://vtiles.example.org"
# export TILE_PATH="/data/basemap/{z}/{x}/{y}.pbf"

export BBOX="-122.52,37.70,-122.35,37.83"   # minLon,minLat,maxLon,maxLat (San Francisco)
export MIN_ZOOM=10
export MAX_ZOOM=16

2. Write the k6 script

The script picks a random zoom, converts a random point in the bounding box to the x/y tile indices at that zoom (the standard slippy-map formula), requests the tile, and checks the response. Thresholds are declared in options so k6 sets a non-zero exit code when they fail — that is what makes the test usable as a CI gate.

import http from 'k6/http';
import { check } from 'k6';
import { Rate } from 'k6/metrics';

const tileErrors = new Rate('tile_errors');

const BASE_URL = __ENV.BASE_URL;
const TILE_PATH = __ENV.TILE_PATH;
const [minLon, minLat, maxLon, maxLat] = __ENV.BBOX.split(',').map(Number);
const MIN_ZOOM = Number(__ENV.MIN_ZOOM || 10);
const MAX_ZOOM = Number(__ENV.MAX_ZOOM || 16);

export const options = {
  // Ramp virtual users so the plateau is a steady load, not a spike.
  stages: [
    { duration: '30s', target: 20 },   // ramp up
    { duration: '2m', target: 20 },    // hold plateau
    { duration: '30s', target: 0 },    // ramp down
  ],
  thresholds: {
    // Fail the run (exit 99) if tail latency or errors exceed budget.
    http_req_duration: ['p(95)<400', 'p(99)<900'],
    tile_errors: ['rate<0.01'],
  },
};

function randInt(min, max) {
  return Math.floor(Math.random() * (max - min + 1)) + min;
}

// Convert a lon/lat point to slippy-map tile x/y at a given zoom.
function lonLatToTile(lon, lat, z) {
  const n = Math.pow(2, z);
  const x = Math.floor(((lon + 180) / 360) * n);
  const latRad = (lat * Math.PI) / 180;
  const y = Math.floor(
    ((1 - Math.log(Math.tan(latRad) + 1 / Math.cos(latRad)) / Math.PI) / 2) * n
  );
  return { x, y };
}

export default function () {
  const z = randInt(MIN_ZOOM, MAX_ZOOM);
  const lon = minLon + Math.random() * (maxLon - minLon);
  const lat = minLat + Math.random() * (maxLat - minLat);
  const { x, y } = lonLatToTile(lon, lat, z);

  const path = TILE_PATH
    .replace('{z}', z)
    .replace('{x}', x)
    .replace('{y}', y);

  const res = http.get(`${BASE_URL}${path}`, {
    tags: { name: 'tile' }, // group all tiles under one metric row
  });

  const ok = check(res, {
    'status is 200': (r) => r.status === 200,
    'body is non-empty': (r) => r.body && r.body.length > 0,
  });
  tileErrors.add(!ok);
}

// Emit a machine-readable summary next to the console output.
export function handleSummary(data) {
  return {
    'summary.json': JSON.stringify(data, null, 2),
    stdout: '\n  see summary.json for full metrics\n',
  };
}

3. Run cache-cold, then cache-warm

Run the identical script twice against the same server. The first run, immediately after flushing the cache, measures the expensive path — reprojection and render-on-miss for MapProxy, cold OS file cache for TileServer GL. The second run, without flushing, measures steady-state warm delivery. The gap between the two is the single most important number for capacity planning.

# Cold run: flush the cache first, then test.
#   MapProxy: restart the cache backend or clear the cache dir/bucket.
k6 run --summary-export cold-summary.json tile-latency.js
mv summary.json cold-summary-full.json

# Warm run: do NOT flush — hit the now-populated cache.
k6 run --summary-export warm-summary.json tile-latency.js
mv summary.json warm-summary-full.json

4. Compare the two servers

Repeat step 3 against the other server by swapping only the two env vars from step 1. Keep every other variable — extent, zoom range, stages, thresholds, and load-generator host — identical, or the comparison is meaningless.

# MapProxy leg
export BASE_URL="https://tiles.example.org"
export TILE_PATH="/tms/1.0.0/parcels/webmercator/{z}/{x}/{y}.png"
k6 run --summary-export mapproxy-warm.json tile-latency.js

# TileServer GL leg (identical extent, zooms, stages)
export BASE_URL="https://vtiles.example.org"
export TILE_PATH="/data/basemap/{z}/{x}/{y}.pbf"
k6 run --summary-export tileserver-warm.json tile-latency.js

Verification

Confirm the run actually exercised the endpoint and that k6 honored the thresholds. A passing run exits 0; a threshold breach exits 99.

# 1. Threshold verdict from the process exit code (99 = a threshold failed).
k6 run tile-latency.js; echo "k6 exit code: $?"
#   k6 exit code: 0        <- all thresholds passed
#   k6 exit code: 99       <- p95/p99 or error-rate budget exceeded

# 2. Read the tail-latency percentiles straight out of the summary.
jq '.metrics.http_req_duration.values | {p95: .["p(95)"], p99: .["p(99)"]}' warm-summary.json
#   { "p95": 118.4, "p99": 233.7 }   <- warm run, milliseconds

# 3. Confirm requests actually succeeded (checks passed, low error rate).
jq '.metrics.tile_errors.values.rate, .root_group.checks' summary.json

# 4. Cold-vs-warm delta — the render-on-miss cost you are sizing for.
jq -n --argjson c "$(cat cold-summary.json)" --argjson w "$(cat warm-summary.json)" \
  '{cold_p95: $c.metrics.http_req_duration.values["p(95)"], warm_p95: $w.metrics.http_req_duration.values["p(95)"]}'

A large cold/warm gap points at expensive cache misses — the case for pre-seeding MapProxy hot extents. A small gap with high absolute latency points at the network or the load generator, not the server. Wire the exit code into a pipeline gate so a regression in tail latency blocks a release, the same discipline covered in Monitoring and Observability for Geospatial Portals.

Troubleshooting matrix

Symptom Likely cause Fix
Every request returns 404 Extent or zoom range has no tiles, or the URL template is wrong Verify BBOX covers real data; open one computed tile URL by hand; check TILE_PATH placeholders
p95 looks great but tile_errors is high check() passing on error pages, or 200s with empty bodies Assert body length and, for vector tiles, content-type application/x-protobuf
Latency dominated by the first few requests TLS handshake / connection setup counted in duration Compare http_req_waiting (server time) instead of total http_req_duration
Cold and warm runs look identical Cache was not actually flushed between runs Confirm the backend cleared; for TileServer GL drop OS page cache or restart the container
Numbers vary wildly between runs Load generator is the bottleneck, or noisy neighbor Run k6 off the server host; raise ulimit; watch generator CPU stays under ~70%
Threshold never fails despite slow tiles Thresholds too loose or metric mis-named Match the metric name exactly (http_req_duration); tighten p(95)/p(99) budgets
429 / connection resets under plateau Reverse-proxy rate limit tripped by the test Whitelist the generator IP or raise the limit for the test window

Up one level: MapProxy vs TileServer GL for Tile Delivery.