SnapshotRate ticks the shard serializes its state and writes it to snapshot storage,
replacing the previous snapshot. On boot the shard loads it back and continues from the tick it was
taken at.
Configuration
SnapshotRate is the amount of play a crash may undo. At 60 TPS, a rate of 300 puts up to five
seconds of the world at risk. NOP storage persists nothing at all — it is the default, and it is
for local development only.
What a snapshot promises, and what it does not
Snapshot writes are asynchronous and best effort. This is deliberate: uploading a world state costs a serialization plus a network round trip, and doing that on the tick goroutine turns every snapshot tick into a latency spike for the whole shard. So the tick hands the snapshot to a background writer and moves on. Three consequences an operator should know:A crash can lose the most recent snapshot
A crash can lose the most recent snapshot
The tick that produced a snapshot returns before the upload finishes. If the process dies in
that window, that snapshot is not in storage — the previous one is. Storage always holds a
complete, valid snapshot (each write replaces the old object atomically); it may just be older
than the last snapshot tick suggests.
Snapshots are latest-wins, never queued
Snapshots are latest-wins, never queued
At most one upload runs at a time. A snapshot produced while an upload is in flight replaces
the one waiting rather than queueing behind it, so a backlog can never grow and the shard cannot
be pushed out of memory by a slow backend. The dropped snapshot is never written — but every
snapshot that IS written is newer than the one before it, so storage never goes backwards.Setting
SnapshotRate faster than the backend can absorb therefore does not make the stored
snapshot more current. It just drops the snapshots in between.An orderly shutdown loses nothing
An orderly shutdown loses nothing
Shutdown takes a final snapshot, then waits for it and anything else outstanding to reach
storage before tearing the shard down. If that wait fails or times out, it is logged at error:
Telling when storage cannot keep up
A dropped snapshot is logged at warn, so it is visible at the default log level:dropped_total is the running count, so gaps between lines are
visible. At shutdown the run’s total is logged once more:
SnapshotRate (snapshot less often) or move to a faster backend. Individual write failures are
reported separately, at warn, as failed to store snapshot.