[tintinhamans]

Moves logging to structured ILogger everywhere and adds OpenTelemetry logs, metrics and traces, exported over OTLP to a collector like Grafana Alloy. Sentry is optional and gets errors, plus logs and traces if enabled. Metrics only go over OTLP since Sentry 6.8 can't read .NET meters. Everything is off by default (OpenTelemetry:Enabled, Sentry:enabled), and docs/observability.md covers config, the metric list and an Alloy example.

Database, HTTP and runtime metrics come from the built-in MySqlConnector, ASP.NET Core, HttpClient and runtime meters. On top of that there are 14 genonline.* metrics worth alerting on: online players, active lobbies, auth attempts, WebSocket connections, failures and send queue depth, matchmaking queue, wait time and matches, background job failures and last success, external publication failures and backlog age, and security actions (rate limits, moderation, anti-cheat). Labels are small fixed sets, never player, lobby or match IDs.

Spans never carry SQL, connection strings or database users, and outbound HTTP spans only keep scheme, host and port since the TURN key is in the Cloudflare path. EF's command events are ignored in the context options, so SQL can't be logged whatever the logging config says. Slow commands (SlowDatabaseCommandMilliseconds, default 500) are logged with operation, duration and result only. Logs and traces do carry user, lobby and match IDs, so treat the collector and Sentry as holding player identifiers.

Builds on #80 (MySqlConnector advisory bump), so its commit stays in the diff until it lands. Lands before #77 and #71–#75, which build on it.

[Skyaero42]

This PR is doing a lot more than logging. Are you sure this PR is in line with the main branch?

[tintinhamans]

This PR is doing a lot more than logging. Are you sure this PR is in line with the main branch?

Was part of the stack before, made it based on main now.

[Skyaero42]

As far as I can tell, this is fine.

[Skyaero42]

For the todo list: once these classes are non-static, logger should be injected through constructor.

[Skyaero42]

I've asked Claude to do a review as well.

I reviewed Services PR 78 and reported 10 findings; I wouldn't merge it as it stands. Telemetry is off by default, so the risk only starts once it's switched on, but several of the alerts the docs recommend would mislead.

Fix before enabling telemetry:

• Connection details in metrics: the database host, port, name and user are exported as the pool.name label on the connection pool metrics. Setting ApplicationName on the connection string builder fixes it.
• enable_logs does nothing: the setting never reaches the logging provider, so ILogger entries never get to Sentry Logs, contrary to the docs.
• WebSocket traces: each connection's messages all land in one trace that ends only at disconnect. Sentry stops adding spans at 1000, so later failures on long sessions go missing.

Metrics that would mislead:

• Backlog age: it grows forever after one permanently failed publication.
• daily_stats_save: it can never report a failure.
• Anti-cheat label: the action is the truncated literal "anticheat_", so bans and review flags look the same.

The other four are lower priority:

• Exception events on spans can still carry SQL text.
• Discord errors are downgraded to Debug.
• EF errors now show up at Error level in Release, including ones the code handles on purpose.
• The send-queue gauge drifts upward.

I checked the Sentry and MySqlConnector behaviour against their source at the pinned versions (6.8.0 and 2.6.2). The findings were not run or reproduced, because I did not build the PR. The review was done from the diff and a read-only clone of the PR head. Nothing was posted to GitHub.