Game server observability for Amazon GameLift Servers

An observability system for studios running dedicated game servers on Amazon GameLift Servers, shipped as telemetry SDKs in C#, C++ and Go, covering Unity and Unreal, on Windows and Linux. I worked on the design of the system as a whole: the metric set, the OpenTelemetry pipeline, the naming and semantics that had to hold identically across three languages, and the dashboards studios get without configuring anything. The binding constraint was the tick: this is extra software on a studio's game backend, and a 60Hz server has about 16ms to do everything in, so the collection could not take a millisecond of it or introduce spikes. I built the Unity SDK myself, end to end, and wrote the public metrics documentation that ships with it. It became the platform's standard observability solution, folded into the main server SDKs so studios install one package rather than two.

• A shared design for the whole system: which metrics exist, what they are called, what they mean, and how they travel, so a studio reading a dashboard sees the same signals whichever language its server is written in.
• OpenTelemetry as the instrumentation layer throughout, keeping studios vendor-neutral and off a bespoke pipeline, with export to Amazon Managed Grafana and CloudWatch.
• Instrumentation written to a hard tick budget: a 60Hz server has roughly 16ms per tick and all of it belongs to the game, so recording a metric had to cost a fraction of a millisecond, on every tick, with no periodic spikes.
• The C# SDK in full: a DogStatsD-style metrics API, server connections, network traffic, CPU, memory and frame timings, with histograms and percentiles for anything latency-shaped.
• A Unity-specific metrics processor to bridge engine realities and the SDK, so measurement stayed accurate without fighting the runtime.
• Parity work across the C++ and Go SDKs and the Unreal integration: reviewing metric semantics, naming and behaviour so the three implementations did not quietly diverge.
• Dashboards that land preconfigured, so the first useful view of a server costs a studio no observability expertise at all.
• Testing against dedicated Linux and Windows server builds, including identifying platform limitations and unsupported deployment targets.
• The public metrics documentation for the plugin.

• Standard rails via OpenTelemetry

  Routing through OpenTelemetry kept game servers on the same observability rails as everything else, rather than creating a second, special system to maintain, and left studios free to point the data wherever they already look.

• One metric contract, three languages

  The names, units and semantics were designed once and then implemented in C#, C++ and Go. Deciding that up front is what makes a single set of dashboards work for every studio, whatever their server is written in.

• The tick budget came first

  We were putting extra software on studios' game backends, and a 60Hz server has about 16ms per tick to do everything in. Observability that eats a millisecond of that is not observability, it is a regression. Variance mattered even more than the average: a collector that is cheap on most ticks and expensive now and then shows up as a frame spike, which is precisely the thing studios install this to hunt. So the hot path stayed as close to free as we could get it, with aggregation and export kept off the tick, and the cost kept flat rather than bursty.

• Histograms over averages

  Percentiles tell you about the bad frames and slow tails that averages hide, which is the entire point on anything latency-sensitive.

• A Unity-aware processor

  Respecting engine constraints with a dedicated processor kept measurement accurate without fighting the runtime.

• One package, not two

  Folding telemetry into the main server SDKs removed an install step and a version-skew problem. An optional second package is a thing studios forget to add, and the metrics nobody installed are worth nothing.

Averages hide exactly the frames you care about. Once the timing signals moved to histograms and percentiles the problems became obvious, and they had been sitting there the whole time. Measuring the right thing beats measuring more things, and on anything latency-sensitive the right thing is almost always the tail. The tick budget taught me the rest: the measurement has to be cheaper than the thing it measures, and cheap on every tick rather than on average, because a spike from the collector is indistinguishable from the bug you were sent to find. Designing for three languages added the last piece: the hard part is not the code, it is agreeing what a metric means and holding that meaning identical across implementations, because the moment two SDKs disagree the dashboard stops being trustworthy. And an SDK nobody can work out how to switch on measures nothing at all, which is why the docs and the default dashboards mattered as much as the instrumentation.