Geospatial Portal

Detecting Terraform Drift in GeoNode Infrastructure

How to catch out-of-band changes to GeoNode’s cloud infrastructure by running a scheduled terraform plan in CI, comparing state to reality, and alerting the moment they diverge.

This page is a hands-on companion to Environment Parity in Geospatial CI Pipelines within the broader Infrastructure Orchestration & Configuration Management practice; read the parent guide for how environments are kept identical, then use this procedure to prove they stay that way. Drift — a security group edited in the console, an RDS parameter tuned by hand, a load balancer someone deleted — silently breaks the guarantee that your code describes your running GeoNode stack. Scheduled drift detection turns that silent divergence into a loud, actionable signal.

Prerequisites

Have these ready before wiring the pipeline, because each is a precondition for a reliable drift signal.

  • Terraform 1.6+ (or OpenTofu 1.6+) with a remote backend — S3 + DynamoDB lock, Terraform Cloud, or GCS — holding the GeoNode stack’s state.
  • A CI runner with read-only cloud credentials scoped to plan; drift detection must never be able to mutate infrastructure.
  • The GeoNode Terraform configuration in version control, applied at least once so state reflects a known-good baseline.
  • Optionally driftctl 0.40+ to catch resources that exist in the cloud but are absent from state (out-of-band creations Terraform cannot see).
  • A notification sink (Slack webhook, PagerDuty, or an issue-tracker token) and permission for CI to open issues or post alerts.

Step-by-step implementation

The core trick is terraform plan -detailed-exitcode, whose exit code is the drift signal: 0 means no changes, 2 means the plan found differences, and 1 means the command errored. A scheduled job runs the plan against locked state, interprets the exit code, and routes a report. The diagram shows that loop.

Scheduled drift-detection loop comparing Terraform state to live GeoNode infrastructure A scheduled trigger, such as an hourly cron, starts a CI job. The job reads the locked remote state and runs terraform plan with the detailed exit code flag using read-only credentials, comparing the recorded state against the real GeoNode infrastructure of RDS, security groups, and load balancers. The command's exit code drives a three-way branch. Exit code zero means state and reality agree, and the job ends green with no drift. Exit code two means the plan detected differences: the job publishes a drift report, alerts the team, and optionally opens an issue or blocks promotion. Exit code one means the command itself errored, which is escalated separately as a pipeline failure. A dashed return arrow shows the loop repeating on the next schedule. schedule hourly cron terraform plan -detailed-exitcode read-only creds locked remote state live infrastructure RDS · SGs · load balancers exit code 0 · 1 · 2 0 · in sync green, no drift 2 · DRIFT report · alert · issue 1 · error escalate pipeline repeats on the next schedule

1. Run a read-only plan with a detailed exit code

The whole detection hinges on one flag. -detailed-exitcode promotes the “changes present” case to exit 2, distinct from a genuine error (1). Run the plan with -refresh-only semantics against locked state, using credentials that cannot apply.

terraform init -input=false -backend-config=backend.hcl

set +e
terraform plan -detailed-exitcode -lock=true -input=false \
  -refresh=true -out=drift.tfplan
code=$?
set -e

case "$code" in
  0) echo "No drift: state matches infrastructure" ;;
  2) echo "DRIFT DETECTED: plan found differences" ;;
  1) echo "ERROR: plan failed" >&2; exit 1 ;;
esac

The -refresh=true step is what actually reads live resource attributes and reconciles them against state, so a hand-edited security group surfaces as a proposed change even though no code changed.

2. Store and compare state artifacts

Capture the machine-readable plan on every run so a human can see what drifted, not just that something did. Convert the binary plan to JSON and keep it as a build artifact keyed by timestamp.

terraform show -json drift.tfplan > drift.json

# Count resources with a non-no-op change (create/update/delete/replace)
drift_count=$(jq '[.resource_changes[]
  | select(.change.actions != ["no-op"])] | length' drift.json)
echo "drifted_resources=$drift_count" >> "$GITHUB_OUTPUT"

Comparing successive drift.json artifacts turns a single alert into a trend — you can see whether the same resource drifts repeatedly, which usually points at automation fighting Terraform.

3. Schedule the check in CI

Run the detection on a cron trigger, not only on pushes, so drift introduced between deploys is caught within the hour. The workflow below is valid GitHub Actions YAML; the schedule and the exit-code interpretation are the load-bearing parts.

name: terraform-drift-detection
on:
  schedule:
    - cron: "0 * * * *"        # hourly drift sweep
  workflow_dispatch: {}

permissions:
  contents: read
  issues: write                 # allow opening a drift issue

jobs:
  drift:
    runs-on: ubuntu-latest
    env:
      AWS_REGION: eu-west-1
      TF_IN_AUTOMATION: "true"
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: "1.9.5"
      - name: Init
        run: terraform init -input=false -backend-config=backend.hcl
      - name: Plan with detailed exit code
        id: plan
        run: |
          set +e
          terraform plan -detailed-exitcode -lock=true -input=false -out=drift.tfplan
          echo "exitcode=$?" >> "$GITHUB_OUTPUT"
      - name: Report drift
        if: steps.plan.outputs.exitcode == '2'
        run: |
          terraform show -no-color drift.tfplan | tee drift-report.txt
          echo "::warning::Terraform drift detected in GeoNode infrastructure"
      - name: Fail on drift
        if: steps.plan.outputs.exitcode == '2'
        run: exit 1
      - name: Fail on error
        if: steps.plan.outputs.exitcode == '1'
        run: exit 1

4. Catch out-of-band resources with driftctl

terraform plan only sees resources Terraform manages. A database or bucket created entirely outside Terraform is invisible to it. driftctl scans the cloud account and diffs it against state, surfacing unmanaged resources — the drift Terraform structurally cannot report.

driftctl scan \
  --from tfstate+s3://geonode-tfstate/prod/terraform.tfstate \
  --output json://driftctl-report.json \
  --output console://

# Non-zero exit if unmanaged or drifted resources exceed the tolerance

Feed the same guardrails you use for environment promotion; the parent workflow that applies this Terraform for GeoNode environments is described in Syncing GeoNode Environments with Terraform.

5. Publish a report, alert, and block promotion

Turn exit 2 into action: post the human-readable plan to chat, open an issue so the drift is tracked, and — for protected environments — make the drift job a required check so a promotion cannot proceed while infrastructure disagrees with code. The GeoNode module whose drift you are watching is typically packaged as HCL like this.

module "geonode" {
  source = "./modules/geonode"

  environment          = "prod"
  rds_instance_class   = "db.r6g.xlarge"
  rds_parameter_group  = aws_db_parameter_group.geonode.name
  alb_security_groups  = [aws_security_group.geonode_alb.id]
  desired_worker_count = 4

  # Lock drift-prone attributes so a console edit is flagged, not absorbed.
  lifecycle_ignore_tags = false
}

Whether this drift gate belongs alongside a Helm-based release or a Kustomize overlay depends on your packaging model, weighed in Helm vs Kustomize for GeoNode Deployments.

Verification

Prove the detector reports 0 on a clean stack and 2 after a deliberate out-of-band change.

# 1. Baseline: a freshly applied stack reports no drift (exit 0)
terraform plan -detailed-exitcode -lock=true; echo "exit=$?"
#   No changes. Your infrastructure matches the configuration.
#   exit=0

# 2. Introduce drift out of band, e.g. edit a security group rule by hand
aws ec2 authorize-security-group-ingress \
  --group-id sg-0geonode --protocol tcp --port 9999 --cidr 0.0.0.0/0

# 3. The next plan must report drift (exit 2) and name the resource
terraform plan -detailed-exitcode -lock=true; echo "exit=$?"
#   ~ aws_security_group.geonode_alb will be updated in place
#   exit=2

# 4. driftctl also catches resources that were never in state
driftctl scan --from tfstate+s3://geonode-tfstate/prod/terraform.tfstate
#   Found N unmanaged resource(s)

An exit=0 on the clean baseline followed by exit=2 after the console edit confirms the pipeline reacts to exactly the changes it should.

Troubleshooting matrix

Symptom Likely cause Fix
Plan always reports drift on every run Provider defaults or computed attributes differ from config Add lifecycle { ignore_changes = [...] } for genuinely computed fields; align defaults
Exit code always 0 despite real changes -detailed-exitcode omitted, so exit collapses to 0/1 Always pass -detailed-exitcode; treat 2 as drift
Scheduled job errors with state-lock timeout A concurrent apply holds the DynamoDB/backend lock Stagger the cron away from deploy windows; use -lock-timeout
Console-created resources never flagged terraform plan only sees managed resources Add driftctl (or import) to surface unmanaged resources
Drift job can mutate infrastructure Runner credentials scoped for apply, not read-only Issue plan-only credentials; deny apply in CI IAM policy
Noise from tag drift on every resource Cost-allocation tags injected out of band Normalize tags via default_tags; ignore the automated tag keys
Alert fires but no one can see what drifted Only the exit code is captured, not the plan Persist terraform show -json as an artifact and post the human-readable plan

For the exact exit-code contract and refresh semantics, the official Terraform plan command reference is authoritative.

Up one level: Environment Parity in Geospatial CI Pipelines.