Performance Telemetry That Can't Identify Anyone

In September I built the pipeline behind CandyRush's public performance page: the game measures how it runs, sends a small summary at the end of each run, a server rolls the sessions up, and the site publishes the result. The design goal was that the privacy page could say something strong and true:

Two sessions from the same player cannot be linked, not by us, and not by anyone reading the stored data.

That sentence holds because of how the system is built, not because of a policy. Four properties carry it:

Property	Where it is enforced
No persistent identifier: a random session id made at launch, thrown away at exit	the payload type in the game
Identifiers are refused, not stripped	the ingest endpoint returns 400
Percentiles per session, then the median across sessions	the roll-up service
Nothing published from fewer than 5 sessions; raw sessions deleted after 5 days	the roll-up service and the storage lifecycle rule

It landed as five merged pull requests in seven days, September 7 to 13, across four repositories:

Pull request	What it did	Merged
CandyRush #873	frame-time histograms, memory and load times, a prod-safe uploader	Sep 7
PublicWebsite #777	the write-only ingest endpoint	Sep 7
SimVerify #44	the roll-up service that turns raw sessions into published windows	Sep 11
PublicWebsite #781	the window picker and schema 2 on the performance page	Sep 11
UTools #90	a shared log scrubber for diagnostic uploads and crash reports	Sep 13

I wrote the issue that split the work into nine steps across two repos, AI coding agents implemented each step, and every pull request went through an adversarial review before I merged it. The interesting parts are the decisions, so that is what follows.

The measurement had to exist in the shipping build

The issue said to add a frame-time histogram to the existing diagnostics reporter. Done literally, that would have produced numbers no player could ever generate: the diagnostics reporters are compiled out of a real production build, and production is exactly the tier a public dashboard is supposed to describe.

So the histograms moved into new, always-compiled telemetry types, and the diagnostics reporters now read from them instead of owning them. The pull request records this as a deliberate deviation from the issue, with the reason.

The histogram itself is fixed-size: 0.5 ms buckets up to 100 ms, 201 integers, the same memory for a five-minute session or a five-hour one. From it the game reports p50, p95, p99 and p99.9, plus the 1% and 0.1% lows.

Two smaller decisions mattered more than they look:

Two histograms, not one. One holds every frame; the other holds only frames rendered while the simulation was running. A single pooled histogram lets a scene-load stall dominate the tail, which says nothing about how combat performs. The all-frames one stays so load stalls remain visible.
"Gameplay frame" is not "a tick ran this frame". At 144 Hz most frames legitimately run zero simulation ticks. The naive rule throws away the fast frames and builds the gameplay distribution almost entirely from slow ones, which is a pessimism bias. The classifier uses a 12-frame grace window and the loop's own pause flag instead.

The payload also carries what the display can present: refresh rate, vsync, and the present cap. Without that, a 40 Hz laptop pools with an uncapped desktop and the published median measures monitors, not the game.

Anonymity as a property of the payload type

The session is identified by a random number generated when the game starts and discarded when it closes. There is no Steam id, no hashed Steam id, no GUID saved to disk, and no hardware fingerprint. The run seed is absent by construction.

Hashing was considered and rejected on purpose. A hashed Steam id is still a stable key: it links every session from the same player, and anyone who has the Steam id can recompute it. That is pseudonymous, not anonymous. The only way to make sessions unlinkable is to never send anything stable.

One implementation detail follows from that. The payload's JSON is written by hand rather than with Unity's JsonUtility, because JsonUtility cannot omit a field. A production payload would have carried "deviceModel":"", and the anonymity claim would have been a convention instead of something the wire format shows.

The server refuses instead of stripping

The ingest endpoint is a Cloudflare Pages Function that only writes. Its checks run in a fixed order: auth, size cap, content type, headers, rate limit, parse, strict per-field validation, then the write to object storage.

Three rules make anonymity checkable at the edge:

A request carrying an X-Steam-Id header is refused, not ignored.
A payload claiming the prod channel while carrying device model, GPU, OS or build id gets a 400, not a quiet strip.
A run seed is refused on every channel.

Why refuse? A server that strips identifiers is a server that receives them. Stripping fails silently: a new field, a renamed field, or a typo in the strip list, and identifying data starts landing in storage with nothing to show for it. A server that refuses fails loudly: the client's upload breaks, and the mistake shows up in testing.

The rate limiter also fails closed. It lives in a KV binding, and a missing binding returns 503, not "unlimited". An endpoint that silently loses its limiter because a binding was forgotten fails in the one way you find out about from a bill.

Raw sessions sit in a private bucket with a five-day lifecycle rule. Five days rather than one is deliberate: a raw session is the one artifact here that cannot be re-derived, so the roll-up needs margin for a failed run.

Rolling up without pooling frames

The roll-up runs in SimVerify, the .NET service that also verifies leaderboard replays. A TelemetryRollupService runs every five minutes per tier: it copies new sessions down from storage, validates each one against the same rules as the endpoint, stores it in SQLite, recomputes the today, yesterday, this-week and all-weeks views, and publishes them with a lastTickUtc heartbeat so the page can tell a stalled roll-up from a quiet day.

The statistics follow from what the data is. Each session already summarizes its own frames, so the roll-up cannot compute a true percentile over all frames pooled together. The page publishes the median across sessions of each session's own percentile, and weights averages by frame count. Any figure drawn from fewer than five sessions is withheld rather than published as noise, and platforms below that floor are left out entirely.

A few details from the pull request:

Device fields stay inside the stored payload JSON and never become database columns, so nothing invites a query that groups by them.
The aggregator is a port of an earlier Python roll-up script. Every Python test was translated to xUnit, and a golden test checks the new service's output against the Python script's output for eight archived dev sessions.
Publishing uses rclone copy, never sync, so a bad local state cannot delete what is already published.

The page, and a 404 that throws

The website change added a window picker (today, yesterday, this week, all weeks, or any past ISO week) and moved the page to the new schema.

The adversarial review's one blocker was easy to miss: the storage host returns its public 404s without CORS headers, so in a browser fetch throws instead of returning a readable 404, and "not published yet" would have shown as an error. The tests now include fetch stubs that throw, the way a real browser does.

The same rule for diagnostic logs

Diagnostic log uploads are a separate route, and the privacy page describes them separately. They needed the same treatment. CandyRush #905 removed the Steam id from the upload's key and metadata, but the log body prints it too, and every save-path string carries the OS account name.

UTools #90 added one scrubber used by two transports: the log uploader and the crash reporter. It replaces user paths, Steam ids and the computer name:

C:\Users\name\...        ->  C:\Users\<user>\...
7656119XXXXXXXXXX        ->  <steamid>
the machine's hostname   ->  <device>

Each rule is narrow on purpose:

Steam ids are matched on their fixed 7656119 prefix, not "any 17 digits", so millisecond timestamps, build ids and frame counters survive.
The hostname is only replaced when it is at least four characters long, so a machine called PC does not shred unrelated text.
Stack-trace file paths are left alone. They are build-machine paths baked in at compile time, not the player's, and rewriting them would break symbolication.

One finding was verified rather than assumed: the crash reporter's event type exposes its breadcrumbs as a read-only collection, so scrubbing them in the BeforeSend hook would have silently done nothing. Each breadcrumb is scrubbed on the way in instead, confirmed by reflecting over the shipped Sentry.dll (4.9.0).

Test counts

Each pull request recorded its own gate run:

Repository	Tests
CandyRush #873	21 of 21 telemetry EditMode tests
PublicWebsite #777	168 of 168 passing, 25 of them new endpoint contract tests
PublicWebsite #781	323 of 323, up from 276
SimVerify #44	1,015 passed, 0 failed, up from 926
UTools #90	16 new scrubber tests

What this does and does not show

The game's shipping configuration has had the uploader switched on since September 7. SimVerify's qa and production roll-up tiers were enabled on September 11, after the dev tier was proven end to end with eight real sessions. This post does not claim anything about production traffic volume; until a window has five sessions, the page shows nothing for it, which is the point.

What generalizes

Make privacy a property of the type, not of the processing. If the payload type cannot hold an identifier, no later code path can leak one.

Refuse, don't strip. A filter fails silently. A refusal fails in testing.

Know what your data can support. Per-session summaries cannot produce a pooled percentile, so the page does not pretend to; it publishes medians across sessions and withholds small samples.