Backup & Disaster Recovery for Spatial Platforms
Backing up a geospatial portal is not the same problem as backing up a database, and treating it as one produces a recovery that restores the data and cannot serve a map. A portal’s recoverable state is spread across a spatial database, an object store of source files, a rendering engine’s own configuration directory, a search index, a message broker, and a tile cache that is expensive rather than irreplaceable — and each of those has a different acceptable loss window and a different restore order. This topic sits inside the Infrastructure Orchestration & Configuration Management framework, and it depends directly on the statefulness model set out there.
The failure this topic exists to prevent is specific and common: a team with working database backups, a documented runbook, and no ability to bring the portal back, because the pieces that were never backed up turn out to be the ones that make the data servable.
What Actually Has to Be Restorable
Sorting the platform’s state by how it can be recovered is the first and most useful exercise, because it tells you what must be backed up, what can be rebuilt, and what can be abandoned.
The third row is the one that catches teams. A rendering engine’s data directory holds workspaces, datastore definitions, styles, layer configuration and security settings — hours of accumulated decisions that exist nowhere else, plus embedded credentials in a recoverable form. A restore that brings back a perfect database into an engine with no workspaces produces a portal that starts, answers, and publishes nothing.
Deciding the Two Numbers That Drive Everything
Every design decision below follows from two numbers, and stating them explicitly turns architecture arguments into arithmetic. The recovery point objective is how much data the organisation can afford to lose, measured in time. The recovery time objective is how long it may take to be serving again.
Write both numbers down per dataset class rather than for the portal as a whole. A cadastral register and a derived heat map do not deserve the same protection, and a single portal-wide objective set from the strictest dataset makes the whole platform more expensive than it needs to be.
Restore Order Is Part of the Plan
A recovery is a sequence, and getting the order wrong produces a portal that comes up in a state nobody expects — layers pointing at a database that is still restoring, a search index built from partial data, workers processing a queue whose tasks refer to rows that do not exist yet.
Step four is deliberately read-only. Bringing the portal back accepting writes before anyone has verified what was restored means new edits land on top of a state nobody has checked, and a subsequent decision to restore to a different point becomes a data-merge problem rather than a repeat of the same procedure.
The Backup That Is Not a Backup
Two arrangements are commonly described as backups and are not, and both are common in geospatial platforms because the data is large and copying it is expensive.
Storage-level replication protects against hardware failure and faithfully replicates a deletion, a bad publish, or an encryption event to the replica within seconds. It is a availability mechanism, not a recovery point. A portal whose only protection is replication cannot go back to yesterday.
A snapshot of a running database volume without coordination captures the files mid-write. Sometimes it restores cleanly; sometimes it restores into a database that starts and reports corruption hours later, under load, in a way that resembles a hardware fault. Use the database’s own backup mechanism, which knows how to produce a consistent copy, rather than a volume snapshot taken on the assumption that the filesystem will sort it out.
The third arrangement worth naming is the backup that has never been restored. It is not a backup either — it is a file with a promising name — and the only way to convert it into a recovery capability is to perform the restore, which is what the running a disaster recovery game day guide exists to make routine.
Verifying a Backup Without Restoring the Whole Portal
A full rehearsal is the real test and is too heavy to run daily. Between rehearsals, a cheap continuous check keeps the gap between “the job succeeded” and “the data is recoverable” from growing unnoticed.
Three checks cover most of the ways a backup silently stops being useful. The first is freshness: the newest restore point is younger than the recovery point objective, asserted against the backup catalogue rather than against the job’s exit code, because a job can succeed while archiving nothing. The second is integrity: the backup system’s own verification pass, which reads the stored blocks and confirms checksums, run on a schedule and alerted on. The third is a partial restore: recover a single small table into a scratch database and compare a row count and a spatial extent against production. That last one takes minutes, exercises the whole path — credentials, network, decryption, the restore tool itself — and is the only one of the three that proves the stored bytes can be turned back into data.
Automate all three and alert on the freshness check most loudly, because it is the one whose failure compounds: an integrity problem is discovered when the check runs, while a stalled archive quietly widens the recovery window every hour until somebody notices.
Getting the Cost Under Control
Spatial data is large, and a naive backup policy for a multi-terabyte portal produces a storage bill that eventually prompts somebody to shorten retention — usually the wrong lever, applied under pressure, without reference to how long a mistake takes to surface.
Three adjustments reduce the cost without reducing the guarantee. Separate the rarely-changing bulk from the frequently-changing core: historic imagery that has not been modified in three years does not need a fresh copy every week, and moving it into a versioned, immutable archive with its own lifecycle rule removes it from the recurring backup entirely. Use incremental backups with periodic full ones, so daily cost tracks the change rate rather than the total size. And apply storage lifecycle transitions deliberately — restore points older than a month can move to slower, cheaper storage provided somebody has checked how long retrieval takes and that the number is compatible with the recovery time objective for that age of restore point.
State the resulting policy in one table alongside the two objectives, so the next cost conversation is about a documented trade-off rather than an improvisation. Anyone can then see that shortening retention from ninety days to thirty is a decision about which class of mistake remains recoverable, not a storage saving with no consequence.
Which Disaster You Are Planning For
“Disaster recovery” covers several very different events, and a plan that treats them as one thing tends to be well prepared for the rarest and unprepared for the most likely. Four scenarios account for nearly everything that actually happens to an agency portal, and they call for different mechanisms.
The most common by a wide margin is human error: a bad publish that replaced a layer with a truncated version, a migration run against the wrong environment, a bulk delete with a filter that matched more than intended. What recovers this is a restore point from before the mistake and the ability to restore a single dataset without taking the portal down — which is a very different capability from restoring the whole platform, and one that a full-platform-only plan does not provide.
Second is component failure: a node lost, a volume corrupted, a database that will not start. Replication and standbys handle this well, and it is the scenario most infrastructure work is implicitly designed around.
Third is data corruption that is discovered late — a geometry repair that quietly invalidated a subset, an import with a coordinate reference mismatch, an attribute overwritten weeks ago. This is the scenario that determines retention, because the interval between the event and its discovery is what the backup window must span, and for spatial data that interval is routinely measured in weeks.
Fourth, and rarest, is loss of the environment: a region gone, an account compromised, infrastructure deleted. This is what cross-region copies, separate credentials and immutable storage exist for, and it is the only scenario where the full sequential recovery is exercised end to end.
Rank them by likelihood for your own portal and check that the plan actually addresses the top of the list. A team that can rebuild the entire platform in another region but cannot restore one layer to yesterday’s state has prepared thoroughly for the least likely event.
Restoring One Dataset Without Restoring Everything
Because human error dominates, the single most valuable recovery capability is also the least dramatic: restoring one layer to a previous state while the portal keeps serving. It needs deliberate preparation, because a database backup is a whole-cluster artefact and extracting one table from it is not a supported operation on any sensible timescale.
Two mechanisms make it practical, and they complement each other. The first is the versioning discipline described in spatial data versioning and rollback workflows: if a publish creates a new version and swaps an alias rather than overwriting in place, undoing a bad publish is an alias swap and needs no backup at all. That is by far the fastest path, and it is available only if the publish pipeline was built for it.
The second is a restore into a scratch database rather than over production. Recover the backup to a temporary instance at the chosen point in time, extract the single table, verify it against what the requester expected, and copy it into place as a new version. It is slower — the restore is the whole cluster even though one table is wanted — but it is always available, it never risks the live data, and it produces a verifiable artefact before anything is changed.
Rehearse the second path even if the first is in place, because the first depends on the pipeline having behaved correctly and the case where you most need a restore is precisely the case where something in the pipeline did not.
Who Decides, and With What Information
Recovery is a decision as much as a procedure, and the decisions are made under time pressure with incomplete information. Naming them in advance removes most of the delay.
Three decisions recur. Which restore point — the most recent, or an earlier one that predates a suspected corruption. Making this well requires knowing when the bad change happened, which is a question for the audit trail rather than for the backup system. Whether to accept the data loss implied by that point, which is a business decision belonging to whoever owns the data rather than to the platform team. And when to reopen writes, which trades the risk of building on an unverified state against the cost of staying read-only.
Record, in the plan, who makes each of those calls and what they need in front of them: the available restore points with their timestamps, the audit trail of recent publishes, and a stated estimate of how long each option takes. A recovery that pauses for an hour while somebody works out who is allowed to accept the loss of a morning’s edits has spent that hour on organisation rather than on technology, and it is a predictable hour.
One more piece of information belongs in front of whoever is deciding: what the portal is currently serving. During a recovery it is easy to lose track of whether readers are seeing stale data, partial data, or an error, and the answer changes which option is acceptable. Put the degraded-mode state and the last successful publish timestamp on the same dashboard as the restore points, so the decision is made against what readers are experiencing rather than against what the recovery scripts are doing.
Operational Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Restore completes but no layers are published | The engine’s configuration directory was never backed up | Back it up as first-class state; treat it as credential-bearing |
| Restored database rejects connections from the engine | Credentials differ between the backup and the restored environment | Re-point datastores through the REST API as part of the restore |
| A restore into staging exposes production data across tenants | Row-level policies were not included in the dump | Include policies, and assert isolation after every restore |
| The restore is far slower than expected | Full backup only, with no incremental or parallel restore | Use incremental backups and parallel restore; measure, do not assume |
| The search index is inconsistent after recovery | Rebuilt while the catalogue was still being restored | Gate the rebuild on the database check passing |
| Duplicate publishes after workers resume | Queue restored rather than reconciled | Drain and reconcile; make tasks idempotent so replay is safe |
| Nobody can find the encryption passphrase | Backup encryption key stored only in the environment being recovered | Store it separately, and verify it can be read as part of the drill |
FAQ
How often should a geospatial portal be backed up?
The database as continuously as the recovery point requires — for most agency portals that means continuous write-ahead archiving with a periodic base backup. Object storage and the engine’s configuration directory daily, because they change slowly and are small. Derived state such as the tile cache and search index not at all, provided the rebuild is exercised and its duration is known.
Should the tile cache be backed up?
Usually not. It is derived, and its value is the render time it saves rather than the data it holds. What should exist instead is a known, measured rebuild path and a fallback for the interval — the serving a degraded basemap during an outage approach covers exactly that window.
Is a cross-region replica enough on its own?
It covers a region failure and nothing else. It faithfully replicates a mistaken deletion, a bad publish, and an encryption event, in seconds. Replication and backup answer different questions and a portal needs both — the replica for availability, the backup for going back in time.
How long should backups be retained?
Long enough to cover the interval between a mistake happening and being noticed, which for a spatial dataset is often weeks rather than days: a wrong attribute value in a rarely-viewed layer can go unreported for a month. Pair a short window of frequent restore points with a longer tail of weekly and monthly copies.
Where should backups be stored?
Somewhere that cannot be reached with the credentials that run the platform, and outside the failure domain being protected against. A backup in the same account, reachable by the same role that a compromised workload holds, is a backup that an incident can delete — which is the scenario immutability and separate credentials exist to prevent.
Related
- Taking Consistent PostGIS Backups with pgBackRest — the database half, done properly.
- Restoring a GeoNode Portal from Cold Backup — the full sequence, executed.
- Replicating Tile Caches Across Regions — keeping the derived tier available.
- Running a Disaster Recovery Game Day — turning a document into a capability.
Up one level: Infrastructure Orchestration & Configuration Management.