Geospatial Portal

Exporting GeoServer Metrics to Prometheus

This guide shows how to expose GeoServer’s JVM, servlet-container, and OGC request metrics to Prometheus by running the JMX exporter as a Java agent, so heap pressure and slow GetMap renders become scrapeable time series.

It is a hands-on companion to Monitoring and Observability for Geospatial Portals and sits within the broader Infrastructure Orchestration & Configuration Management practice; read the parent guide first if you need the rationale for which GeoServer signals matter and why the RED method frames the request path. Here the focus is purely mechanical: attach the exporter, whitelist the right beans, expose a port, and confirm the series land in Prometheus.

Prerequisites

  • GeoServer 2.22+ running in Tomcat or the official docker.osgeo.org/geoserver image, with access to set JAVA_OPTS or CATALINA_OPTS.
  • The jmx_prometheus_javaagent JAR (0.20.0 or newer) from the Prometheus JMX exporter project, placed on a volume the GeoServer container can read.
  • Prometheus 2.45+, or the Prometheus Operator (kube-prometheus-stack) if you scrape via a ServiceMonitor.
  • Permission to edit the GeoServer Deployment/Pod spec and to add a scrape target or ServiceMonitor in the monitoring namespace.
  • Network reachability from Prometheus to the GeoServer metrics port (default 9404), not blocked by a NetworkPolicy.

The topology below shows the agent living inside the GeoServer JVM, translating JMX beans into a Prometheus exposition endpoint that Prometheus scrapes and Grafana reads.

GeoServer JVM to JMX exporter to Prometheus to Grafana scrape path Inside the GeoServer JVM, the servlet container and OGC dispatcher publish MBeans covering heap, garbage collection, threads, and request counts. The jmx_prometheus_javaagent runs in-process, reads those MBeans through the whitelist, and serves them as Prometheus text on port 9404. Prometheus scrapes that endpoint every fifteen seconds and stores the series, and Grafana queries Prometheus to draw the JVM and request panels. GEOSERVER JVM MBeans heap · GC · threads Tomcat · requests OGC dispatcher GetMap · GetFeature JMX exporter -javaagent whitelist rules :9404 /metrics Prometheus scrape_interval 15s stores series scrape Grafana JVM · request panels query

Step-by-step implementation

1. Place the agent and write its config

The JMX exporter runs in-process as a -javaagent, so it needs no separate JVM and sees every MBean GeoServer publishes. Mount the JAR and a config file into the container. The config below whitelists JVM, Tomcat thread-pool, and GeoServer request beans and drops everything else, which keeps cardinality bounded.

# jmx-exporter-config.yaml — mounted at /opt/jmx/config.yaml
startDelaySeconds: 0
lowercaseOutputName: true
lowercaseOutputLabelNames: true
# Only export the beans we actually chart; everything else is ignored.
whitelistObjectNames:
  - "java.lang:type=Memory,*"
  - "java.lang:type=GarbageCollector,*"
  - "java.lang:type=Threading,*"
  - "Catalina:type=ThreadPool,*"
  - "Catalina:type=GlobalRequestProcessor,*"
  - "org.geoserver:*"
rules:
  # Tomcat request processor -> request rate + latency labels
  - pattern: 'Catalina<type=GlobalRequestProcessor, name="(.+)"><>(\w+):'
    name: geoserver_tomcat_$2
    labels:
      processor: "$1"
    type: COUNTER
  # Catch-all for remaining whitelisted beans
  - pattern: ".*"

Keeping the config as a file rather than inline flags means the whitelist is version-controlled with the rest of the deployment, the same parity discipline described in Environment Parity in Geospatial CI Pipelines.

2. Attach the agent to the GeoServer JVM

Add the agent to JAVA_OPTS. The argument is port:configfile — the port the exporter listens on and the path to the config from step 1.

# Appended to CATALINA_OPTS / JAVA_OPTS for the GeoServer process
export JAVA_OPTS="$JAVA_OPTS \
  -javaagent:/opt/jmx/jmx_prometheus_javaagent-0.20.0.jar=9404:/opt/jmx/config.yaml"

In Kubernetes, set this through the pod’s environment and mount the JAR and config from a ConfigMap and an init-container-populated volume:

env:
  - name: EXTRA_JAVA_OPTS
    value: "-javaagent:/opt/jmx/jmx_prometheus_javaagent-0.20.0.jar=9404:/opt/jmx/config.yaml"

3. Expose the metrics port

Declare the port on the container and on the Service so Prometheus has a stable target. Give it a named port (metrics) — the ServiceMonitor in step 4 selects it by name.

apiVersion: v1
kind: Service
metadata:
  name: geoserver
  namespace: geoportal
  labels:
    app: geoserver
spec:
  selector:
    app: geoserver
  ports:
    - name: http
      port: 8080
      targetPort: 8080
    - name: metrics
      port: 9404
      targetPort: 9404

4. Add a Prometheus scrape target

If you run plain Prometheus, add a scrape_config:

scrape_configs:
  - job_name: geoserver
    metrics_path: /metrics
    scrape_interval: 15s
    static_configs:
      - targets: ["geoserver.geoportal.svc.cluster.local:9404"]
        labels:
          component: geoserver

If you run the Prometheus Operator, ship a ServiceMonitor instead — it selects the Service by label and the port by name:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: geoserver
  namespace: geoportal
  labels:
    release: kube-prometheus-stack
spec:
  selector:
    matchLabels:
      app: geoserver
  endpoints:
    - port: metrics
      interval: 15s
      path: /metrics

5. Import a Grafana panel

With the series flowing, add a panel that reads GeoServer heap headroom and a companion that reads request latency. Grafana panels are JSON; the fragment below defines a single time-series panel targeting the exporter’s heap metric.

{
  "title": "GeoServer JVM heap used",
  "type": "timeseries",
  "datasource": { "type": "prometheus", "uid": "prometheus" },
  "targets": [
    {
      "expr": "jvm_memory_bytes_used{area=\"heap\",component=\"geoserver\"}",
      "legendFormat": "heap used"
    }
  ],
  "fieldConfig": { "defaults": { "unit": "bytes" } }
}

Pair it with a latency panel using histogram_quantile(0.99, sum by (le) (rate(geoserver_tomcat_requestprocessingtime_bucket[5m]))) so heap pressure and slow renders sit side by side on the GeoServer board described in the parent guide.

What the JVM tells you that the request metrics do not

Request-level metrics answer whether the renderer is slow; JVM metrics answer why, and for a Java OGC engine the answer is usually one of three things. Heap pressure shows up as rising garbage-collection time that steals wall-clock from every request simultaneously, so latency degrades uniformly rather than for a particular layer. Thread-pool exhaustion shows up as requests queueing before they are ever executed, so the engine’s own timing looks healthy while the client waits. Connection-pool exhaustion at the data source shows up as requests blocked in the pool, which is invisible in both of the previous views.

Three internal saturation modes and their external signature Three cards: heap, threads, and data source pool, each with the metric that moves, the external symptom, and the metric that would mislead you. Heap pressure watch GC pause time per minute old-gen occupancy after GC looks like every layer slows together, in steps, not smoothly misleads: CPU looks busy Thread pool watch busy vs max threads accept queue length looks like clients time out while the engine reports fast requests misleads: internal p99 is fine Data source pool watch connections in use vs max time spent waiting for one looks like slow “queries” the database has no record of running misleads: database looks idle

Scrape all three from the same endpoint and put them on one row of the dashboard, in that order, because that is the order in which they should be ruled out. The third card is the one that sends teams to the wrong team: the database is genuinely idle, its slow-query log is genuinely empty, and the requests are genuinely waiting — inside the engine, for a connection that the pool has not got.

Verification

Confirm each hop before trusting the dashboard.

# 1. The exporter is serving metrics from inside the pod
kubectl exec -n geoportal deploy/geoserver -- \
  curl -s localhost:9404/metrics | grep -c jvm_memory_bytes_used
#   expect: a non-zero count (heap gauges present)

# 2. Key GeoServer/Tomcat series are exposed, not just JVM defaults
kubectl exec -n geoportal deploy/geoserver -- \
  curl -s localhost:9404/metrics | grep geoserver_tomcat_

# 3. Prometheus considers the target up
curl -s 'http://prometheus.geoportal:9090/api/v1/query?query=up{job="geoserver"}' \
  | grep -o '"value":\[[^]]*\]'
#   expect: ... ,"1"]  (1 = target healthy)

# 4. A real request metric increments after driving traffic
curl -s 'http://prometheus.geoportal:9090/api/v1/query?query=jvm_threads_current{component="geoserver"}'

A non-zero heap gauge and an up value of 1 confirm the agent is attached and Prometheus is scraping. If geoserver_tomcat_ series are missing while JVM series appear, the whitelist or the rule pattern is the culprit, not the scrape.

Scrape interval, histograms, and what they cost

Two configuration choices decide whether these metrics are useful or merely present. The first is scrape interval. A fifteen-second scrape cannot describe a ten-second incident, and tile incidents are often that short — a seeding burst, a garbage-collection pause, a failover. But halving the interval doubles the storage and the load of every series, and for a JVM exporter with hundreds of series per instance that is not free. The workable compromise is a short interval for the small set of series that describe user experience and a longer one for everything else, achieved with separate scrape jobs rather than a single global setting.

The second is histogram bucket layout. A latency histogram is only as informative as its bucket boundaries, and the defaults shipped by most exporters are tuned for web applications: buckets clustered under a second, with the highest boundary somewhere around ten. Tile rendering has a different distribution — most responses land in tens of milliseconds when cached, and uncached deep-zoom renders can take several seconds — so default buckets put nearly every observation in one or two of them and the resulting quantiles are noise dressed as precision.

Bucket boundaries decide whether a quantile means anything Two rows of bucket boundaries under the same distribution: default buckets crowd the observations into one bucket, tile-tuned buckets spread them across several. Default buckets — almost everything lands in one < 0.1s: 94% 0.25s 0.5s 1s 10s p95 can only be reported as “somewhere under 0.1s” — the interesting tail is unresolved Buckets chosen for tile traffic — the tail is visible 5ms 15ms 40ms 100ms 300ms 800ms 2s 5s+ cached hits and uncached renders separate cleanly, so a rising p95 names which population moved

Pick boundaries from a day of observed latencies rather than from a template, and revisit them after any change that shifts the distribution — a new cache tier, a projection change, a hardware move. Bucket boundaries are part of the measurement, and changing them resets history, so make the choice once, deliberately, and record why.

Troubleshooting matrix

Symptom Likely cause Fix
GeoServer fails to start after adding the agent Wrong JAR path or Java version mismatch in -javaagent Verify the path exists in the container and the JAR matches the JVM major version; check the container startup log
/metrics returns connection refused Agent argument malformed — missing port:configfile form Confirm the flag is =9404:/opt/jmx/config.yaml, not two separate args
Only jvm_* series appear, no geoserver_* whitelistObjectNames omits the GeoServer/Catalina beans Add the Catalina:* and org.geoserver:* object names and reload
Prometheus target shows down NetworkPolicy blocks port 9404, or wrong Service port name Allow ingress from the monitoring namespace to 9404; confirm the metrics port name matches the ServiceMonitor
Metrics present but cardinality explodes, Prometheus RAM climbs A rule maps a high-cardinality label (per-layer or per-URL) Tighten the pattern rules; drop labels that carry unbounded values
histogram_quantile returns empty The latency metric is a gauge, not a histogram with _bucket series Confirm the bean exposes buckets; otherwise chart the gauge directly

Up one level: Monitoring and Observability for Geospatial Portals.