Sealing GeoNode Secrets for GitOps

This guide keeps the portal’s credentials in the same version-controlled repository as its manifests, encrypted so that only the destination cluster can read them. It belongs to Secrets Management for Geospatial Platforms, within the Core Portal Architecture & Security Boundaries framework.

Prerequisites

  • A GitOps reconciler already syncing the portal’s manifests, as set up in promoting GeoNode releases with Argo CD.
  • Cluster-admin rights to install a sealing controller, and one namespace per environment.
  • A backup destination for the controller’s private key that is not the cluster it protects.
  • The current plaintext values, so the migration can be verified before the old copies are removed.

The property that makes this work

Sealing is asymmetric encryption with the private half held only by a controller in the destination cluster. Anyone can encrypt a value for that cluster; only that cluster can decrypt it. That is what makes the ciphertext safe to commit — and it is also what makes the private key the single most consequential artefact in the arrangement.

From plaintext to a committed ciphertext to a running Secret Four stages — encrypt with the public key, commit, reconcile, decrypt in-cluster — with a note that the private key never leaves the cluster. 1 · encrypt with the cluster's public key, on a laptop 2 · commit ciphertext lives beside the manifests 3 · reconcile applied like any other resource 4 · decrypt in cluster controller holds the private key What this does and does not give you Gives: no plaintext in version control, one review flow for everything, and a deployment that needs no external store at run time. Does not give: rotation machinery, an access audit trail, or protection from anyone who can read Secrets in the namespace. The ciphertext is also permanent: a value committed today is in the history forever, so a leak of the private key is retroactive.

That last line is the property people underestimate. Git history is durable and widely replicated, so a sealed value committed once can be decrypted by anyone who later obtains the private key — including from a backup of the cluster taken years afterwards. Sealing protects the repository; it does not make the value disposable.

Step-by-step implementation

1. Install the controller and record its public key

# Install the sealing controller into its own namespace.
kubectl create namespace sealed-secrets
helm upgrade --install sealed-secrets sealed-secrets/sealed-secrets \
  --namespace sealed-secrets \
  --set fullnameOverride=sealed-secrets-controller

# Fetch the public certificate and commit it — it is public by design, and
# committing it means anyone can seal a value without cluster access.
kubeseal --controller-namespace sealed-secrets --fetch-cert \
  > clusters/production/sealing-cert.pem

2. Back up the private key before anything depends on it

This step is skipped more often than any other and is the one that makes a cluster rebuild survivable. Without the key, every sealed value in the repository is unreadable and must be re-created from sources that may no longer exist.

# The controller stores its keys as Secrets labelled for this purpose.
kubectl -n sealed-secrets get secret \
  -l sealedsecrets.bitnami.com/sealed-secrets-key -o yaml \
  > sealing-keys-backup.yaml

# Store this OUTSIDE the cluster it protects, encrypted, with the same handling
# rules as any other catastrophic-class credential. Then verify you can read it.
age -r "$OFFLINE_RECIPIENT" -o sealing-keys-backup.yaml.age sealing-keys-backup.yaml
shred -u sealing-keys-backup.yaml

3. Seal the portal’s values

Scope each sealed value to the namespace and name it will be applied as. The default strict scope means a sealed value cannot be moved to another namespace or renamed — which is a feature: it stops a secret intended for staging being applied in production.

# Create the Secret locally, seal it, and never write the plaintext to disk.
kubectl create secret generic geonode-env \
  --namespace geoportal \
  --from-literal=DATABASE_URL="postgres://geonode_app:${DB_PASSWORD}@postgis:5432/geoportal" \
  --from-literal=SECRET_KEY="${DJANGO_SECRET_KEY}" \
  --dry-run=client -o yaml \
| kubeseal --cert clusters/production/sealing-cert.pem --format yaml \
  > clusters/production/geoportal/geonode-env.sealed.yaml

4. Commit the ciphertext and let the reconciler apply it

# clusters/production/geoportal/geonode-env.sealed.yaml (excerpt)
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
  name: geonode-env
  namespace: geoportal
spec:
  encryptedData:
    DATABASE_URL: AgBv8xR2...        # ciphertext, safe to commit
    SECRET_KEY: AgCz1kQ9...
  template:
    metadata:
      name: geonode-env
      namespace: geoportal
    type: Opaque

5. Remove the plaintext copies you were relying on before

The migration is not finished when the sealed value works. It is finished when the previous copies are gone: the manifest that used to carry the value, the entry in a shared password manager that was the working copy, and the developer shell history that created it. Until then the credential still exists in the places the migration was meant to remove.

Rotation, which is where the model shows its cost

Rotation: sealed ciphertext versus an external store Two rows comparing the steps required to rotate a credential under each model, and what remains afterwards. Sealed in the repository re-encrypt commit review merge reconcile and afterwards: the superseded ciphertext stays in history, decryptable by anyone who ever obtains the private key Held in an external store update the value consumers reload no repository change, no review queue, nothing left behind Sealing is the right choice when a store is unavailable or the deployment must be self-contained — not when rotation is frequent.

The comparison is not an argument against sealing. It is an argument for choosing it deliberately: sealing suits values that change rarely and must travel with the manifests — a service account’s credential, an application signing key — and suits frequently rotated database credentials much less well. Many portals end up with both, which is fine provided each value’s home is a decision rather than an accident.

One key per environment, and what that buys

Sharing a key pair across environments is the shortcut that makes staging and production interchangeable — which is exactly the property you do not want for credentials.

Shared sealing key versus one key per environment Two rows comparing what a mistaken apply does and how far a key exposure reaches under each arrangement. One shared key simpler to manage a staging value applies cleanly in production one exposure reaches every environment at once avoid One key per environment one certificate each a cross-environment apply simply fails to unseal an exposure is contained to one environment prefer Name the certificates after the cluster they belong to, and keep each beside that cluster's manifests rather than in a shared folder.

The failure this prevents is mundane and common: an engineer copies a sealed file between environment folders to save re-sealing, it works, and six months later nobody can say whether production is running the staging credential or the other way round.

Verification

# 1. The sealed resource decrypts into a working Secret
kubectl -n geoportal get sealedsecret geonode-env -o jsonpath='{.status.conditions[0].type}'
#   expect: Synced
kubectl -n geoportal get secret geonode-env -o jsonpath='{.data.SECRET_KEY}' | wc -c
#   expect: a non-zero length

# 2. The repository contains no plaintext
git grep -nE '(DATABASE_URL|SECRET_KEY)\s*[:=]\s*["'\'']?[A-Za-z0-9+/]{12,}' -- clusters/ || echo "clean"
#   expect: clean

# 3. The sealed value cannot be applied to another namespace
sed 's/namespace: geoportal/namespace: default/' \
  clusters/production/geoportal/geonode-env.sealed.yaml | kubectl apply -f -
#   expect: the controller refuses to unseal — strict scope is working

# 4. The private key backup can actually be read
age -d -i "$OFFLINE_IDENTITY" sealing-keys-backup.yaml.age | head -3
#   expect: valid YAML, not a decryption error

# 5. The portal starts with the sealed values
kubectl -n geoportal rollout status deploy/geonode --timeout=180s
#   expect: successfully rolled out

Check 4 is the one that is easy to skip and expensive to skip. A backup that has never been decrypted is a file, not a recovery path, and the moment it is needed is the worst moment to discover which passphrase was used.

Troubleshooting matrix

Symptom Likely cause Fix
no key could decrypt secret after a cluster rebuild The controller generated a new key pair; old ciphertext is unreadable Restore the backed-up keys before reconciling, or re-seal every value
Sealed value works in staging but not production Sealed against the wrong cluster’s certificate Keep one certificate per cluster in the repository and name them clearly
The Secret exists but the pod still uses old values The pod was not restarted after the Secret changed Roll the deployment, or add a checksum annotation so a change forces a restart
A reviewer cannot tell what changed in a sealed file Ciphertext diffs are meaningless by design Require the pull request to state which keys changed and why
Controller pod crash-loops after upgrade Key format or CRD version mismatch Pin the controller version alongside the sealed resources; upgrade in staging first
A leaked key means every historical value is exposed Ciphertext is permanent in Git history Rotate every value sealed with that key, not only the current ones
Sealing works but nothing is auditable The model has no access log — the cluster reads the Secret directly Use an external store for values where read auditing is a requirement

FAQ

Is committing encrypted secrets really safe?

Safe enough for values that change rarely, provided the private key is protected and backed up outside the cluster, and provided you accept that the ciphertext is permanent. The realistic threat is not that someone breaks the encryption; it is that the private key is later exposed through a cluster backup, and every value ever sealed becomes readable at once.

How does this compare with an external secret store?

Sealing removes a run-time dependency and keeps everything in one review flow, which is genuinely valuable for a small team. An external store adds rotation machinery and an access audit trail, which matter more as the platform grows and as compliance obligations arrive. The two coexist comfortably: seal what is stable and self-contained, store what rotates or must be audited.

What happens when the cluster is rebuilt?

Everything sealed for the old cluster becomes unreadable unless the controller’s keys are restored first. That single fact is why the backup step is not optional and why the restore is worth rehearsing — a rebuild is precisely the moment when nobody wants to discover that the portal’s entire configuration must be re-created from memory.

Should each environment have its own key?

Yes. Separate keys are what make it impossible to apply a production value to staging, and they limit the blast radius of a key exposure to one environment. The cost is one more certificate to manage per environment, which is small next to the property it buys.

Can a sealed secret be rotated without a pull request?

Not within this model, and that is the trade-off. If a credential needs to be rotatable under incident conditions — quickly, without a review queue, possibly while the pipeline itself is broken — it belongs in an external store instead. Deciding that per credential class, rather than for the platform as a whole, is what keeps both mechanisms honest.

Up one level: Secrets Management for Geospatial Platforms.