Skip to content

Chunked replay upload

The active online run path uses the existing combat replay model and rule-based validator, but transports replay data as independently compressed chunks. Inline replays in rpc_submit_run are rejected.

Format

  • Protocol/schema: replay-v2-chunked
  • Payload schema after reconstruction: existing ReplayPayload schema 1
  • Serialization: compact camel-case JSON, UTF-8
  • Compression: one independent Gzip member per chunk
  • Chunk indices: contiguous and zero-based
  • Target compressed chunk size: 384 KiB
  • Maximum compressed chunk size: 512 KiB
  • Maximum chunks: 128
  • Maximum uncompressed replay size: 64 MiB
  • Maximum compressed replay size: 32 MiB
  • Existing semantic ceilings remain 65,536 events and 65,536 checkpoints
  • Client upload parallelism: 3
  • Per-chunk timeout: 30 seconds
  • Retry limit: 5 with exponential backoff starting at 250 ms

Each decompressed chunk is:

{
  "schemaVersion": "replay-v2-chunked",
  "chunkIndex": 0,
  "startedAtTick": 100,
  "completedAtTick": 200,
  "events": [],
  "checkpoints": []
}

The compressed hash is SHA-256 over the UTF-8 bytes of the canonical base64 representation. This representation is used because Nakama's sandboxed JavaScript API hashes strings, not byte arrays. The uncompressed hash is SHA-256 over the exact UTF-8 segment JSON. The full hash is SHA-256 over the concatenated uncompressed segment JSON bytes in ascending chunk order. Gzip decompression uses .NET's BCL on the client and bundled pako on Nakama 3.22, whose JavaScript runtime has no native Gzip API.

Manifest

Server collection replay_manifests, key <replayId>, owner <playerId>, owner-readable and server-write-only:

{
  "protocol": "replay-v2-chunked",
  "replayId": "<sha256>",
  "runId": "...",
  "submissionId": "<sha256>",
  "playerId": "...",
  "uploadSessionId": "<server uuid>",
  "seedId": "...",
  "scoreVersion": "performance-score-v2",
  "replayFormatVersion": "replay-v2-chunked",
  "gameVersion": "0.1.0",
  "buildVersion": "development",
  "balanceVersion": "...",
  "runDefinitionHash": "...",
  "loadoutHash": "...",
  "startedAtTick": 100,
  "completedAtTick": 200,
  "durationMs": 100,
  "chunkCount": 3,
  "totalEventCount": 900,
  "uncompressedSizeBytes": 800000,
  "compressedSizeBytes": 300000,
  "fullHash": "<sha256>",
  "status": "Uploading",
  "createdAt": "...",
  "updatedAt": "...",
  "expiresAt": "...",
  "completedAt": "...",
  "lastError": "..."
}

Statuses are Created, Uploading, Finalizing, Complete, Failed, Expired, and Rejected. Terminal replays cannot be modified.

Chunks use collection replay_chunks, key <replayId>:<zero-padded-index>. They contain replay/run/owner identity, index, base64 Gzip data, compressed and uncompressed hashes and sizes, and upload time. Both collections have permissionWrite=0; other players cannot overwrite them.

RPC flow

  1. replay_upload_begin binds an immutable manifest to the authenticated run, submission, seed, versions, ticket, and owner. Repeating the same request returns the existing server-generated uploadSessionId, present indices, 512 KiB server limit, and expiry.
  2. replay_upload_status returns uploaded/missing indices, stored bytes, status, expiry, and last error.
  3. replay_chunk_upload accepts chunks in any order. An identical duplicate is successful; the same index with different content is rejected. The server validates token, index, size, base64 hash, Gzip, uncompressed hash, segment schema, segment index, timeline, and record limits before storage.
  4. replay_upload_finalize verifies every stored chunk again, aggregate sizes, order, full hash, schema, and record count, then marks the manifest Complete. Missing chunks leave it resumable. Repeated finalization is safe.
  5. rpc_submit_run accepts only replayId and replayHash, reconstructs the completed replay through the shared server reader, executes the existing authoritative rule validation and score calculation, and only then writes history and a leaderboard projection.

The client queries status before upload and schedules only missing indices. There is never more than one task for an index. Cancellation propagates through parallel upload, delay, and timeout waits.

Local files and resume

Production uses user://replays/<replayId>/:

manifest.json
chunk-0000.gz
chunk-0001.gz
upload-state.json

upload-state.json stores submission and upload-session IDs, confirmed zero-based indices, status, retry count, last error, creation time, and whether the server confirmation was persisted. The package reader verifies disk data before use. A retry (including a package loaded after restart) begins idempotently, obtains status, and skips confirmed server chunks.

Local cleanup retains at most 8 replay directories, 128 MiB total, and 14 days. The active replay is protected. A replay is removed only after server finalization and accepted canonical result confirmation have been written locally.

Server cleanup and retention

cleanupReplayUploadsForOwner idempotently removes expired incomplete, failed/rejected, and orphaned records for an owner, but never Complete manifests. The authenticated maintenance RPC is owner-scoped; production-wide cleanup requires trusted orchestration over owners. Completed replay artifacts used by run history retain the existing 180-day audit marker and must not be deleted while referenced by leaderboard, dispute, or audit policy.

Logs contain correlation fields (runId, submissionId, replayId, playerId, and chunk index) for begin, duplicate/store/reject, resume, finalization, validation, and cleanup. Replay contents and upload tokens are never logged.

Testing interruption and resume

  1. Start Nakama and complete a run.
  2. Disconnect the client after one or more ReplayChunkStored log entries.
  3. Reconnect and trigger pending-result retry.
  4. Confirm replay_upload_status reports uploaded and missing indices.
  5. Confirm only missing indices produce new ReplayChunkStored entries.
  6. Disconnect after all chunks but before finalization; retry and verify idempotent finalization.
  7. Verify no leaderboard row exists before ReplayUploadFinalized and RunValidationAccepted.

Automated coverage is in ChunkedReplayTests.cs and modules/tests/chunkedReplay.test.js.

Schema migration

Version changes require a new protocol identifier, reader branch, client/server deployment, and compatibility fixtures. Do not reinterpret stored bytes under an existing schema. Existing replay-json-v1 inline artifacts remain readable through their legacy artifact path but cannot be submitted through the new online flow. In-progress v1 client submissions must be completed before rollout or discarded and replayed; there is no lossless migration from a missing local v1 payload.

Current boundary

Recording uses the existing event/checkpoint model. Mid-run, the recorder seals immutable event prefixes through ChunkedReplayChunkBuilder once they reach the target compressed size (checkpoints and power-ups stay local until finalize so packing order matches ChunkedReplayCodec.Encode).

Sealed chunks are enqueued on ReplayUploadQueue and uploaded in the background via replay_stream_begin / replay_stream_chunk under a provisional streaming replay id (SHA256("stream:" + runId + ":" + playerId)). The recorder never waits on network I/O.

At successful run end the client drains the queue, finalizes the remaining package, then uses the existing replay_upload_begin → missing replay_chunk_upload → replay_upload_finalize → rpc_submit_run path. Begin adopts already-streamed chunks with matching hashes so only the tail (if any) is uploaded after the run.

Aborted runs cancel the streaming queue, call replay_stream_abort (also invoked from rpc_abort_run), and never upload/validate or project a leaderboard row. Expired Streaming manifests are removed by replay_upload_cleanup_owned.

Chunk segments use chunk-local tick ranges (within the manifest span). Power-up selections ride in a dedicated trailing chunk so mid-run sealed prefixes stay byte-stable.