Skip to content

ci: expand Python SDK harness protocol and codec coverage - #929

Draft
dustinbyrne wants to merge 1 commit into
mainfrom
ci/sdk-harness-coverage-2026-09-07
Draft

ci: expand Python SDK harness protocol and codec coverage#929
dustinbyrne wants to merge 1 commit into
mainfrom
ci/sdk-harness-coverage-2026-09-07

Conversation

@dustinbyrne

@dustinbyrne dustinbyrne commented Sep 7, 2026

Copy link
Copy Markdown
Contributor

💡 Motivation and Context

Expand recurring Python SDK coverage to harness 1.0.0, including UTC timestamps, dedicated AI capture and supported V1 codecs. The four asynchronous profiles use the real posthog.Client: V0 gzip, V1 gzip, V1 deflate and V1 zstd.

  • Map compression enable/disable through existing public capture_compression, install the optional zstd extra in V1 adapter builds, and advertise the selected codec.
  • Preserve the SDK's native GeoIP default and use public unbounded flush for adapter drain operations.
  • Keep compliance assertions advisory, run every capability-selected suite and upload distinct profile artifacts.

No shipped SDK source, public API, defaults or package dependencies change. This is adapter/CI-only work and does not require a package release.

💚 How did you test it?

Built and installed the worktree wheel with the zstd extra in isolated Python 3.12.12. Ran the unchanged pinned harness against loopback, with all selected suites and sequential execution:

Profile Selected Passed Failed
V0 gzip 52 51 1
V1 gzip 117 115 2
V1 deflate 116 114 2
V1 zstd 116 114 2

These retain two known SDK/harness disagreements: feature_flags.request_payload.disable_geoip_omitted_defaults_to_false expects false instead of Python's native true; V1 also fails capture_ai.routing.capture_does_not_reroute_ai_named_events, whose assertion hardcodes /batch for ordinary analytics capture.

All 16 adapter regression tests pass, including real SDK packets independently decoded as gzip, deflate and zstd. Repository Ruff lint/format, scoped mypy, public API snapshot and installed-package import checks pass. Docker was unavailable locally; the profile builds and image-backed reports are the CI validation gate.

The pinned mock cannot decode full deflate/zstd traffic or compressed partial responses. Codec header passes are not delivery certificates; ordinary profile traffic is uncompressed unless a test enables compression. Sync-mode applicability and existing diagnostic-state limitations remain documented gaps.

📝 Checklist

  • I reviewed the submitted code.
  • I added tests to verify the changes.
  • I updated the docs if needed.
  • No breaking change or entry added to the changelog.

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

Implemented with the Pi coding agent using repository inspection, local Python tooling and GitHub CLI. A fresh read-only agent reviewed the same commit and found no issues. Human review is still required; no manual end-user testing is claimed.

@dustinbyrne dustinbyrne self-assigned this Sep 7, 2026
@github-actions

github-actions Bot commented Sep 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

posthog-python Compliance Report

Date: 2026-09-07T20:18:19.901683+00:00
Duration: 208196ms

⚠️ Some Tests Failed

115/117 tests passed, 2 failed

Capture_V1 Tests

✅ 95/95 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint ✅ 15ms
Endpoint And Method.Does Not Use Legacy Endpoints ✅ 9ms
Required Headers.Has Authorization Bearer Header ✅ 8ms
Required Headers.Has Content Type Json ✅ 7ms
Required Headers.Has Posthog Sdk Info Format ✅ 8ms
Required Headers.Has Posthog Attempt Header ✅ 7ms
Required Headers.Has Posthog Request Id ✅ 7ms
Required Headers.Has Posthog Request Timestamp ✅ 8ms
Required Headers.Has User Agent ✅ 7ms
Body Format.Body Has Created At And Batch ✅ 7ms
Body Format.No Api Key In Body ✅ 9ms
Body Format.No Sent At In Body ✅ 7ms
Event Format.Event Has Required Root Fields ✅ 7ms
Event Format.Event Uuid Is Valid ✅ 7ms
Event Format.Event Timestamp Is Rfc3339 ✅ 8ms
Event Format.Non Utc Event Timestamp Is Converted To Utc ✅ 12ms
Event Format.Distinct Id Is String ✅ 7ms
Event Format.Distinct Id At Root Not Properties ✅ 8ms
Event Format.Custom Properties Preserved ✅ 7ms
Event Format.Set Properties Preserved ✅ 7ms
Event Format.Set Once Properties Preserved ✅ 8ms
Event Format.Groups Properties Preserved ✅ 7ms
Event Format.Sdk Generates Uuid If Not Provided ✅ 7ms
Event Format.Event Has Required Root Fields Batch ✅ 11ms
Event Format.Event Uuid Is Valid Batch ✅ 9ms
Event Format.Event Timestamp Is Rfc3339 Batch ✅ 10ms
Event Format.Distinct Id Is String Batch ✅ 10ms
Event Format.Distinct Id At Root Not Properties Batch ✅ 10ms
Event Format.Custom Properties Preserved Batch ✅ 9ms
Event Format.Set Properties Preserved Batch ✅ 11ms
Event Format.Set Once Properties Preserved Batch ✅ 10ms
Event Format.Groups Properties Preserved Batch ✅ 10ms
Event Format.Sdk Generates Uuid If Not Provided Batch ✅ 10ms
Batch Behavior.Multiple Events In Single Batch ✅ 14ms
Batch Behavior.Batch Envelope Smoke ✅ 11ms
Batch Behavior.Flush With No Events Sends Nothing ✅ 5ms
Batch Behavior.Flush At Triggers Batch ✅ 1008ms
Batch Behavior.Created At Reflects Batch Creation Time ✅ 11ms
Deduplication.Generates Unique Uuids ✅ 17ms
Deduplication.Different Events Same Content Different Uuids ✅ 10ms
Deduplication.Preserves Uuid On Retry ✅ 6016ms
Deduplication.Preserves Timestamp On Retry ✅ 6017ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 6025ms
Deduplication.No Duplicate Events In Batch ✅ 20ms
Header Behavior On Retry.Attempt Header Starts At One ✅ 9ms
Header Behavior On Retry.Attempt Header Increments On Retry ✅ 13027ms
Header Behavior On Retry.Request Id Preserved On Retry ✅ 6015ms
Header Behavior On Retry.Different Requests Have Different Request Ids ✅ 2020ms
Header Behavior On Retry.Request Timestamp Changes On Retry ✅ 6021ms
Response Format Validation.Success Response Has Uuid Keyed Results ✅ 10ms
Response Format Validation.Success Response Has Ok For Each Event ✅ 11ms
Response Format Validation.Success No Retry After When All Ok ✅ 9ms
Response Format Validation.Success Retry After Present When Retry Events ✅ 1012ms
Response Format Validation.Success No Retry After When Drop Only ✅ 12ms
Response Format Validation.Response Echoes Request Id ✅ 7ms
Retry Behavior.Retries On 408 ✅ 6016ms
Retry Behavior.Retries On 500 ✅ 6021ms
Retry Behavior.Retries On 503 ✅ 8021ms
Retry Behavior.Retries On 504 ✅ 6020ms
Retry Behavior.Retryable Errors Have Retry After ✅ 3018ms
Retry Behavior.Respects Retry After On Retryable Error ✅ 11024ms
Retry Behavior.Does Not Retry On 400 ✅ 2014ms
Retry Behavior.Does Not Retry On 401 ✅ 2012ms
Retry Behavior.Does Not Retry On 402 ✅ 2014ms
Retry Behavior.Does Not Retry On 413 ✅ 2014ms
Retry Behavior.Does Not Retry On 415 ✅ 2013ms
Retry Behavior.Non Retryable Errors Have No Retry After ✅ 2014ms
Retry Behavior.Implements Backoff ✅ 22037ms
Retry Behavior.Max Retries Respected ✅ 22032ms
Partial Batch Handling.Handles 200 Full Success ✅ 2012ms
Partial Batch Handling.Handles 200 With All Ok ✅ 3018ms
Partial Batch Handling.Does Not Retry Dropped Events ✅ 3018ms
Partial Batch Handling.Does Not Retry Limited Events ✅ 3017ms
Partial Batch Handling.Prunes Ok Events On Partial Retry ✅ 6017ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry ✅ 6022ms
Partial Batch Handling.Retries Only Retry Events From Partial ✅ 6021ms
Partial Batch Handling.Partial Retry Preserves Uuids ✅ 6021ms
Partial Batch Handling.Partial Retry Attempt Header Increments ✅ 6023ms
Partial Batch Handling.Partial Retry Request Id Preserved ✅ 6021ms
Partial Batch Handling.Respects Retry After On Partial ✅ 8023ms
Partial Batch Handling.Unknown Result Treated As Terminal ✅ 3016ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry ✅ 3019ms
Compression.Sends Gzip Content Encoding ✅ 10ms
Compression.No Content Encoding When Disabled ✅ 9ms
Compression.Compressed Body Is Decompressible ✅ 10ms
Error Handling.Does Not Retry On Unknown 4Xx ✅ 2010ms
Event Options.Cookieless Mode Override ✅ 11ms
Event Options.Disable Skew Correction Override ✅ 9ms
Event Options.Process Person Profile Override ✅ 8ms
Event Options.Product Tour Id Override ✅ 9ms
Event Options.Unset Options Omitted ✅ 9ms
Event Options.Options Override In Batch ✅ 10ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties ✅ 7ms
Geoip And Historical Migration.Historical Migration Set In Body ✅ 7ms
Geoip And Historical Migration.Historical Migration Absent By Default ✅ 6ms

Capture_Ai Tests

⚠️ 4/5 tests passed, 1 failed

View Details
Test Status Duration
Routing.Capture Ai Posts To Ai Endpoint ✅ 7ms
Routing.Capture Does Not Reroute Ai Named Events ❌ 7ms
Identity.Capture Ai Event Has Uuid ✅ 7ms
Identity.Capture Ai Keeps Supplied Uuid ✅ 6ms
Timestamp Format.Non Utc Event Timestamp Is Converted To Utc ✅ 7ms

Failures

routing.capture_does_not_reroute_ai_named_events

Request 0 path '/i/v1/analytics/events' != expected '/batch'

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 8ms
Request Payload.Flags Request Uses V2 Query Param ✅ 8ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 7ms
Request Payload.Flags Request Omits Authorization Header ✅ 8ms
Request Payload.Token In Flags Body Matches Init ✅ 7ms
Request Payload.Groups Round Trip ✅ 8ms
Request Payload.Groups Default To Empty Object ✅ 7ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 7ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 8ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 7ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 7ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 10ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 8ms
Retry Behavior.Retries Flags On 502 ✅ 312ms
Retry Behavior.Retries Flags On 504 ✅ 313ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 10ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Expected geoip_disable=False, got True

@dustinbyrne

dustinbyrne commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

SDK compliance report — all CI profiles

SDK head: 16ef8bb3b90995e5afa1499d180d8c56be53ff15
CI checkout: 5c114317724952231ee001a66db007648ce823c7
Harness: 1.0.0 / contract 1.2.

These reports are from the completed CI artifacts linked below. All selected results are included; assertion failures remain advisory and are not hidden by successful workflow checks.

Profile Passed / selected Failed Evidence
V0gzip 51/52 1 CI run
V1gzip 115/117 2 CI run
V1deflate 114/116 2 CI run
V1zstd 114/116 2 CI run

All four profiles use the actual async SDK. Native GeoIP defaults and ordinary AI-named capture routing disagreements remain visible. Non-gzip cases do not certify full compressed-body delivery because the mock cannot decode those bodies.

V0gzip: 51/52 passed, 1 failed — full report

posthog-python Compliance Report

Date: 2026-09-07T20:16:27.341665+00:00
Duration: 118556ms

⚠️ Some Tests Failed

51/52 tests passed, 1 failed

Capture Tests

✅ 30/30 tests passed

View Details
Test Status Duration
Format Validation.Event Has Required Fields ✅ 15ms
Format Validation.Event Has Uuid ✅ 9ms
Format Validation.Event Has Lib Properties ✅ 9ms
Format Validation.Distinct Id Is String ✅ 8ms
Format Validation.Token Is Present ✅ 9ms
Format Validation.Custom Properties Preserved ✅ 8ms
Format Validation.Event Has Timestamp ✅ 9ms
Format Validation.Non Utc Event Timestamp Is Converted To Utc ✅ 15ms
Retry Behavior.Retries On 503 ✅ 8018ms
Retry Behavior.Does Not Retry On 400 ✅ 2012ms
Retry Behavior.Does Not Retry On 401 ✅ 2011ms
Retry Behavior.Respects Retry After Header ✅ 8017ms
Retry Behavior.Implements Backoff ✅ 22032ms
Retry Behavior.Retries On 500 ✅ 6018ms
Retry Behavior.Retries On 502 ✅ 6017ms
Retry Behavior.Retries On 504 ✅ 6015ms
Retry Behavior.Max Retries Respected ✅ 22033ms
Deduplication.Generates Unique Uuids ✅ 17ms
Deduplication.Preserves Uuid On Retry ✅ 6012ms
Deduplication.Preserves Uuid And Timestamp On Retry ✅ 13025ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 6021ms
Deduplication.No Duplicate Events In Batch ✅ 18ms
Deduplication.Different Events Have Different Uuids ✅ 10ms
Compression.Sends Gzip When Enabled ✅ 9ms
Batch Format.Uses Proper Batch Structure ✅ 9ms
Batch Format.Flush With No Events Sends Nothing ✅ 5ms
Batch Format.Multiple Events Batched Together ✅ 15ms
Error Handling.Does Not Retry On 403 ✅ 2010ms
Error Handling.Does Not Retry On 413 ✅ 2011ms
Error Handling.Retries On 408 ✅ 6016ms

Capture_Ai Tests

✅ 5/5 tests passed

View Details
Test Status Duration
Routing.Capture Ai Posts To Ai Endpoint ✅ 11ms
Routing.Capture Does Not Reroute Ai Named Events ✅ 8ms
Identity.Capture Ai Event Has Uuid ✅ 8ms
Identity.Capture Ai Keeps Supplied Uuid ✅ 9ms
Timestamp Format.Non Utc Event Timestamp Is Converted To Utc ✅ 8ms

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 9ms
Request Payload.Flags Request Uses V2 Query Param ✅ 9ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 9ms
Request Payload.Flags Request Omits Authorization Header ✅ 9ms
Request Payload.Token In Flags Body Matches Init ✅ 8ms
Request Payload.Groups Round Trip ✅ 10ms
Request Payload.Groups Default To Empty Object ✅ 9ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 9ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 9ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 10ms
Request Lifecycle.No Flags Request On Init Alone ✅ 4ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 10ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 13ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 10ms
Retry Behavior.Retries Flags On 502 ✅ 312ms
Retry Behavior.Retries Flags On 504 ✅ 312ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 10ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Expected geoip_disable=False, got True
V1gzip: 115/117 passed, 2 failed — full report

posthog-python Compliance Report

Date: 2026-09-07T20:18:19.901683+00:00
Duration: 208196ms

⚠️ Some Tests Failed

115/117 tests passed, 2 failed

Capture_V1 Tests

✅ 95/95 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint ✅ 15ms
Endpoint And Method.Does Not Use Legacy Endpoints ✅ 9ms
Required Headers.Has Authorization Bearer Header ✅ 8ms
Required Headers.Has Content Type Json ✅ 7ms
Required Headers.Has Posthog Sdk Info Format ✅ 8ms
Required Headers.Has Posthog Attempt Header ✅ 7ms
Required Headers.Has Posthog Request Id ✅ 7ms
Required Headers.Has Posthog Request Timestamp ✅ 8ms
Required Headers.Has User Agent ✅ 7ms
Body Format.Body Has Created At And Batch ✅ 7ms
Body Format.No Api Key In Body ✅ 9ms
Body Format.No Sent At In Body ✅ 7ms
Event Format.Event Has Required Root Fields ✅ 7ms
Event Format.Event Uuid Is Valid ✅ 7ms
Event Format.Event Timestamp Is Rfc3339 ✅ 8ms
Event Format.Non Utc Event Timestamp Is Converted To Utc ✅ 12ms
Event Format.Distinct Id Is String ✅ 7ms
Event Format.Distinct Id At Root Not Properties ✅ 8ms
Event Format.Custom Properties Preserved ✅ 7ms
Event Format.Set Properties Preserved ✅ 7ms
Event Format.Set Once Properties Preserved ✅ 8ms
Event Format.Groups Properties Preserved ✅ 7ms
Event Format.Sdk Generates Uuid If Not Provided ✅ 7ms
Event Format.Event Has Required Root Fields Batch ✅ 11ms
Event Format.Event Uuid Is Valid Batch ✅ 9ms
Event Format.Event Timestamp Is Rfc3339 Batch ✅ 10ms
Event Format.Distinct Id Is String Batch ✅ 10ms
Event Format.Distinct Id At Root Not Properties Batch ✅ 10ms
Event Format.Custom Properties Preserved Batch ✅ 9ms
Event Format.Set Properties Preserved Batch ✅ 11ms
Event Format.Set Once Properties Preserved Batch ✅ 10ms
Event Format.Groups Properties Preserved Batch ✅ 10ms
Event Format.Sdk Generates Uuid If Not Provided Batch ✅ 10ms
Batch Behavior.Multiple Events In Single Batch ✅ 14ms
Batch Behavior.Batch Envelope Smoke ✅ 11ms
Batch Behavior.Flush With No Events Sends Nothing ✅ 5ms
Batch Behavior.Flush At Triggers Batch ✅ 1008ms
Batch Behavior.Created At Reflects Batch Creation Time ✅ 11ms
Deduplication.Generates Unique Uuids ✅ 17ms
Deduplication.Different Events Same Content Different Uuids ✅ 10ms
Deduplication.Preserves Uuid On Retry ✅ 6016ms
Deduplication.Preserves Timestamp On Retry ✅ 6017ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 6025ms
Deduplication.No Duplicate Events In Batch ✅ 20ms
Header Behavior On Retry.Attempt Header Starts At One ✅ 9ms
Header Behavior On Retry.Attempt Header Increments On Retry ✅ 13027ms
Header Behavior On Retry.Request Id Preserved On Retry ✅ 6015ms
Header Behavior On Retry.Different Requests Have Different Request Ids ✅ 2020ms
Header Behavior On Retry.Request Timestamp Changes On Retry ✅ 6021ms
Response Format Validation.Success Response Has Uuid Keyed Results ✅ 10ms
Response Format Validation.Success Response Has Ok For Each Event ✅ 11ms
Response Format Validation.Success No Retry After When All Ok ✅ 9ms
Response Format Validation.Success Retry After Present When Retry Events ✅ 1012ms
Response Format Validation.Success No Retry After When Drop Only ✅ 12ms
Response Format Validation.Response Echoes Request Id ✅ 7ms
Retry Behavior.Retries On 408 ✅ 6016ms
Retry Behavior.Retries On 500 ✅ 6021ms
Retry Behavior.Retries On 503 ✅ 8021ms
Retry Behavior.Retries On 504 ✅ 6020ms
Retry Behavior.Retryable Errors Have Retry After ✅ 3018ms
Retry Behavior.Respects Retry After On Retryable Error ✅ 11024ms
Retry Behavior.Does Not Retry On 400 ✅ 2014ms
Retry Behavior.Does Not Retry On 401 ✅ 2012ms
Retry Behavior.Does Not Retry On 402 ✅ 2014ms
Retry Behavior.Does Not Retry On 413 ✅ 2014ms
Retry Behavior.Does Not Retry On 415 ✅ 2013ms
Retry Behavior.Non Retryable Errors Have No Retry After ✅ 2014ms
Retry Behavior.Implements Backoff ✅ 22037ms
Retry Behavior.Max Retries Respected ✅ 22032ms
Partial Batch Handling.Handles 200 Full Success ✅ 2012ms
Partial Batch Handling.Handles 200 With All Ok ✅ 3018ms
Partial Batch Handling.Does Not Retry Dropped Events ✅ 3018ms
Partial Batch Handling.Does Not Retry Limited Events ✅ 3017ms
Partial Batch Handling.Prunes Ok Events On Partial Retry ✅ 6017ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry ✅ 6022ms
Partial Batch Handling.Retries Only Retry Events From Partial ✅ 6021ms
Partial Batch Handling.Partial Retry Preserves Uuids ✅ 6021ms
Partial Batch Handling.Partial Retry Attempt Header Increments ✅ 6023ms
Partial Batch Handling.Partial Retry Request Id Preserved ✅ 6021ms
Partial Batch Handling.Respects Retry After On Partial ✅ 8023ms
Partial Batch Handling.Unknown Result Treated As Terminal ✅ 3016ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry ✅ 3019ms
Compression.Sends Gzip Content Encoding ✅ 10ms
Compression.No Content Encoding When Disabled ✅ 9ms
Compression.Compressed Body Is Decompressible ✅ 10ms
Error Handling.Does Not Retry On Unknown 4Xx ✅ 2010ms
Event Options.Cookieless Mode Override ✅ 11ms
Event Options.Disable Skew Correction Override ✅ 9ms
Event Options.Process Person Profile Override ✅ 8ms
Event Options.Product Tour Id Override ✅ 9ms
Event Options.Unset Options Omitted ✅ 9ms
Event Options.Options Override In Batch ✅ 10ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties ✅ 7ms
Geoip And Historical Migration.Historical Migration Set In Body ✅ 7ms
Geoip And Historical Migration.Historical Migration Absent By Default ✅ 6ms

Capture_Ai Tests

⚠️ 4/5 tests passed, 1 failed

View Details
Test Status Duration
Routing.Capture Ai Posts To Ai Endpoint ✅ 7ms
Routing.Capture Does Not Reroute Ai Named Events ❌ 7ms
Identity.Capture Ai Event Has Uuid ✅ 7ms
Identity.Capture Ai Keeps Supplied Uuid ✅ 6ms
Timestamp Format.Non Utc Event Timestamp Is Converted To Utc ✅ 7ms

Failures

routing.capture_does_not_reroute_ai_named_events

Request 0 path '/i/v1/analytics/events' != expected '/batch'

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 8ms
Request Payload.Flags Request Uses V2 Query Param ✅ 8ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 7ms
Request Payload.Flags Request Omits Authorization Header ✅ 8ms
Request Payload.Token In Flags Body Matches Init ✅ 7ms
Request Payload.Groups Round Trip ✅ 8ms
Request Payload.Groups Default To Empty Object ✅ 7ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 7ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 8ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 7ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 7ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 10ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 8ms
Retry Behavior.Retries Flags On 502 ✅ 312ms
Retry Behavior.Retries Flags On 504 ✅ 313ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 10ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Expected geoip_disable=False, got True
V1deflate: 114/116 passed, 2 failed — full report

posthog-python Compliance Report

Date: 2026-09-07T20:18:08.970708+00:00
Duration: 208204ms

⚠️ Some Tests Failed

114/116 tests passed, 2 failed

Capture_V1 Tests

✅ 94/94 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint ✅ 15ms
Endpoint And Method.Does Not Use Legacy Endpoints ✅ 9ms
Required Headers.Has Authorization Bearer Header ✅ 8ms
Required Headers.Has Content Type Json ✅ 7ms
Required Headers.Has Posthog Sdk Info Format ✅ 8ms
Required Headers.Has Posthog Attempt Header ✅ 8ms
Required Headers.Has Posthog Request Id ✅ 8ms
Required Headers.Has Posthog Request Timestamp ✅ 7ms
Required Headers.Has User Agent ✅ 8ms
Body Format.Body Has Created At And Batch ✅ 7ms
Body Format.No Api Key In Body ✅ 9ms
Body Format.No Sent At In Body ✅ 7ms
Event Format.Event Has Required Root Fields ✅ 8ms
Event Format.Event Uuid Is Valid ✅ 7ms
Event Format.Event Timestamp Is Rfc3339 ✅ 8ms
Event Format.Non Utc Event Timestamp Is Converted To Utc ✅ 12ms
Event Format.Distinct Id Is String ✅ 7ms
Event Format.Distinct Id At Root Not Properties ✅ 7ms
Event Format.Custom Properties Preserved ✅ 8ms
Event Format.Set Properties Preserved ✅ 8ms
Event Format.Set Once Properties Preserved ✅ 7ms
Event Format.Groups Properties Preserved ✅ 8ms
Event Format.Sdk Generates Uuid If Not Provided ✅ 7ms
Event Format.Event Has Required Root Fields Batch ✅ 10ms
Event Format.Event Uuid Is Valid Batch ✅ 11ms
Event Format.Event Timestamp Is Rfc3339 Batch ✅ 11ms
Event Format.Distinct Id Is String Batch ✅ 11ms
Event Format.Distinct Id At Root Not Properties Batch ✅ 10ms
Event Format.Custom Properties Preserved Batch ✅ 11ms
Event Format.Set Properties Preserved Batch ✅ 11ms
Event Format.Set Once Properties Preserved Batch ✅ 10ms
Event Format.Groups Properties Preserved Batch ✅ 11ms
Event Format.Sdk Generates Uuid If Not Provided Batch ✅ 11ms
Batch Behavior.Multiple Events In Single Batch ✅ 14ms
Batch Behavior.Batch Envelope Smoke ✅ 12ms
Batch Behavior.Flush With No Events Sends Nothing ✅ 5ms
Batch Behavior.Flush At Triggers Batch ✅ 1009ms
Batch Behavior.Created At Reflects Batch Creation Time ✅ 10ms
Deduplication.Generates Unique Uuids ✅ 15ms
Deduplication.Different Events Same Content Different Uuids ✅ 10ms
Deduplication.Preserves Uuid On Retry ✅ 6012ms
Deduplication.Preserves Timestamp On Retry ✅ 6016ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 6022ms
Deduplication.No Duplicate Events In Batch ✅ 17ms
Header Behavior On Retry.Attempt Header Starts At One ✅ 8ms
Header Behavior On Retry.Attempt Header Increments On Retry ✅ 13025ms
Header Behavior On Retry.Request Id Preserved On Retry ✅ 6019ms
Header Behavior On Retry.Different Requests Have Different Request Ids ✅ 2019ms
Header Behavior On Retry.Request Timestamp Changes On Retry ✅ 6019ms
Response Format Validation.Success Response Has Uuid Keyed Results ✅ 11ms
Response Format Validation.Success Response Has Ok For Each Event ✅ 12ms
Response Format Validation.Success No Retry After When All Ok ✅ 10ms
Response Format Validation.Success Retry After Present When Retry Events ✅ 1014ms
Response Format Validation.Success No Retry After When Drop Only ✅ 11ms
Response Format Validation.Response Echoes Request Id ✅ 8ms
Retry Behavior.Retries On 408 ✅ 6017ms
Retry Behavior.Retries On 500 ✅ 6015ms
Retry Behavior.Retries On 503 ✅ 8023ms
Retry Behavior.Retries On 504 ✅ 6019ms
Retry Behavior.Retryable Errors Have Retry After ✅ 3017ms
Retry Behavior.Respects Retry After On Retryable Error ✅ 11022ms
Retry Behavior.Does Not Retry On 400 ✅ 2013ms
Retry Behavior.Does Not Retry On 401 ✅ 2014ms
Retry Behavior.Does Not Retry On 402 ✅ 2013ms
Retry Behavior.Does Not Retry On 413 ✅ 2012ms
Retry Behavior.Does Not Retry On 415 ✅ 2012ms
Retry Behavior.Non Retryable Errors Have No Retry After ✅ 2013ms
Retry Behavior.Implements Backoff ✅ 22035ms
Retry Behavior.Max Retries Respected ✅ 22036ms
Partial Batch Handling.Handles 200 Full Success ✅ 2011ms
Partial Batch Handling.Handles 200 With All Ok ✅ 3016ms
Partial Batch Handling.Does Not Retry Dropped Events ✅ 3017ms
Partial Batch Handling.Does Not Retry Limited Events ✅ 3015ms
Partial Batch Handling.Prunes Ok Events On Partial Retry ✅ 6020ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry ✅ 6022ms
Partial Batch Handling.Retries Only Retry Events From Partial ✅ 6022ms
Partial Batch Handling.Partial Retry Preserves Uuids ✅ 6021ms
Partial Batch Handling.Partial Retry Attempt Header Increments ✅ 6018ms
Partial Batch Handling.Partial Retry Request Id Preserved ✅ 6021ms
Partial Batch Handling.Respects Retry After On Partial ✅ 8017ms
Partial Batch Handling.Unknown Result Treated As Terminal ✅ 3016ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry ✅ 3019ms
Compression.Sends Deflate Content Encoding ✅ 11ms
Compression.No Content Encoding When Disabled ✅ 8ms
Error Handling.Does Not Retry On Unknown 4Xx ✅ 2011ms
Event Options.Cookieless Mode Override ✅ 10ms
Event Options.Disable Skew Correction Override ✅ 8ms
Event Options.Process Person Profile Override ✅ 8ms
Event Options.Product Tour Id Override ✅ 8ms
Event Options.Unset Options Omitted ✅ 8ms
Event Options.Options Override In Batch ✅ 11ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties ✅ 7ms
Geoip And Historical Migration.Historical Migration Set In Body ✅ 8ms
Geoip And Historical Migration.Historical Migration Absent By Default ✅ 7ms

Capture_Ai Tests

⚠️ 4/5 tests passed, 1 failed

View Details
Test Status Duration
Routing.Capture Ai Posts To Ai Endpoint ✅ 8ms
Routing.Capture Does Not Reroute Ai Named Events ❌ 8ms
Identity.Capture Ai Event Has Uuid ✅ 7ms
Identity.Capture Ai Keeps Supplied Uuid ✅ 8ms
Timestamp Format.Non Utc Event Timestamp Is Converted To Utc ✅ 7ms

Failures

routing.capture_does_not_reroute_ai_named_events

Request 0 path '/i/v1/analytics/events' != expected '/batch'

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 9ms
Request Payload.Flags Request Uses V2 Query Param ✅ 9ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 8ms
Request Payload.Flags Request Omits Authorization Header ✅ 9ms
Request Payload.Token In Flags Body Matches Init ✅ 8ms
Request Payload.Groups Round Trip ✅ 9ms
Request Payload.Groups Default To Empty Object ✅ 8ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 9ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 8ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 8ms
Request Lifecycle.No Flags Request On Init Alone ✅ 3ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 10ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 11ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 8ms
Retry Behavior.Retries Flags On 502 ✅ 313ms
Retry Behavior.Retries Flags On 504 ✅ 313ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 12ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Expected geoip_disable=False, got True
V1zstd: 114/116 passed, 2 failed — full report

posthog-python Compliance Report

Date: 2026-09-07T20:18:07.813432+00:00
Duration: 208341ms

⚠️ Some Tests Failed

114/116 tests passed, 2 failed

Capture_V1 Tests

✅ 94/94 tests passed

View Details
Test Status Duration
Endpoint And Method.Targets V1 Endpoint ✅ 16ms
Endpoint And Method.Does Not Use Legacy Endpoints ✅ 9ms
Required Headers.Has Authorization Bearer Header ✅ 9ms
Required Headers.Has Content Type Json ✅ 8ms
Required Headers.Has Posthog Sdk Info Format ✅ 9ms
Required Headers.Has Posthog Attempt Header ✅ 8ms
Required Headers.Has Posthog Request Id ✅ 9ms
Required Headers.Has Posthog Request Timestamp ✅ 8ms
Required Headers.Has User Agent ✅ 8ms
Body Format.Body Has Created At And Batch ✅ 8ms
Body Format.No Api Key In Body ✅ 9ms
Body Format.No Sent At In Body ✅ 9ms
Event Format.Event Has Required Root Fields ✅ 8ms
Event Format.Event Uuid Is Valid ✅ 8ms
Event Format.Event Timestamp Is Rfc3339 ✅ 9ms
Event Format.Non Utc Event Timestamp Is Converted To Utc ✅ 14ms
Event Format.Distinct Id Is String ✅ 9ms
Event Format.Distinct Id At Root Not Properties ✅ 8ms
Event Format.Custom Properties Preserved ✅ 8ms
Event Format.Set Properties Preserved ✅ 9ms
Event Format.Set Once Properties Preserved ✅ 8ms
Event Format.Groups Properties Preserved ✅ 8ms
Event Format.Sdk Generates Uuid If Not Provided ✅ 8ms
Event Format.Event Has Required Root Fields Batch ✅ 12ms
Event Format.Event Uuid Is Valid Batch ✅ 12ms
Event Format.Event Timestamp Is Rfc3339 Batch ✅ 13ms
Event Format.Distinct Id Is String Batch ✅ 11ms
Event Format.Distinct Id At Root Not Properties Batch ✅ 12ms
Event Format.Custom Properties Preserved Batch ✅ 12ms
Event Format.Set Properties Preserved Batch ✅ 12ms
Event Format.Set Once Properties Preserved Batch ✅ 11ms
Event Format.Groups Properties Preserved Batch ✅ 12ms
Event Format.Sdk Generates Uuid If Not Provided Batch ✅ 12ms
Batch Behavior.Multiple Events In Single Batch ✅ 16ms
Batch Behavior.Batch Envelope Smoke ✅ 13ms
Batch Behavior.Flush With No Events Sends Nothing ✅ 6ms
Batch Behavior.Flush At Triggers Batch ✅ 1010ms
Batch Behavior.Created At Reflects Batch Creation Time ✅ 10ms
Deduplication.Generates Unique Uuids ✅ 16ms
Deduplication.Different Events Same Content Different Uuids ✅ 11ms
Deduplication.Preserves Uuid On Retry ✅ 6017ms
Deduplication.Preserves Timestamp On Retry ✅ 6015ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry ✅ 6023ms
Deduplication.No Duplicate Events In Batch ✅ 17ms
Header Behavior On Retry.Attempt Header Starts At One ✅ 8ms
Header Behavior On Retry.Attempt Header Increments On Retry ✅ 13026ms
Header Behavior On Retry.Request Id Preserved On Retry ✅ 6016ms
Header Behavior On Retry.Different Requests Have Different Request Ids ✅ 2018ms
Header Behavior On Retry.Request Timestamp Changes On Retry ✅ 6019ms
Response Format Validation.Success Response Has Uuid Keyed Results ✅ 11ms
Response Format Validation.Success Response Has Ok For Each Event ✅ 12ms
Response Format Validation.Success No Retry After When All Ok ✅ 10ms
Response Format Validation.Success Retry After Present When Retry Events ✅ 1016ms
Response Format Validation.Success No Retry After When Drop Only ✅ 12ms
Response Format Validation.Response Echoes Request Id ✅ 8ms
Retry Behavior.Retries On 408 ✅ 6017ms
Retry Behavior.Retries On 500 ✅ 6013ms
Retry Behavior.Retries On 503 ✅ 8022ms
Retry Behavior.Retries On 504 ✅ 6019ms
Retry Behavior.Retryable Errors Have Retry After ✅ 3017ms
Retry Behavior.Respects Retry After On Retryable Error ✅ 11022ms
Retry Behavior.Does Not Retry On 400 ✅ 2013ms
Retry Behavior.Does Not Retry On 401 ✅ 2014ms
Retry Behavior.Does Not Retry On 402 ✅ 2011ms
Retry Behavior.Does Not Retry On 413 ✅ 2013ms
Retry Behavior.Does Not Retry On 415 ✅ 2012ms
Retry Behavior.Non Retryable Errors Have No Retry After ✅ 2012ms
Retry Behavior.Implements Backoff ✅ 22025ms
Retry Behavior.Max Retries Respected ✅ 22036ms
Partial Batch Handling.Handles 200 Full Success ✅ 2012ms
Partial Batch Handling.Handles 200 With All Ok ✅ 3017ms
Partial Batch Handling.Does Not Retry Dropped Events ✅ 3015ms
Partial Batch Handling.Does Not Retry Limited Events ✅ 3013ms
Partial Batch Handling.Prunes Ok Events On Partial Retry ✅ 6021ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry ✅ 6017ms
Partial Batch Handling.Retries Only Retry Events From Partial ✅ 6022ms
Partial Batch Handling.Partial Retry Preserves Uuids ✅ 6020ms
Partial Batch Handling.Partial Retry Attempt Header Increments ✅ 6020ms
Partial Batch Handling.Partial Retry Request Id Preserved ✅ 6021ms
Partial Batch Handling.Respects Retry After On Partial ✅ 8021ms
Partial Batch Handling.Unknown Result Treated As Terminal ✅ 3013ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry ✅ 3018ms
Compression.Sends Zstd Content Encoding ✅ 9ms
Compression.No Content Encoding When Disabled ✅ 9ms
Error Handling.Does Not Retry On Unknown 4Xx ✅ 2009ms
Event Options.Cookieless Mode Override ✅ 10ms
Event Options.Disable Skew Correction Override ✅ 8ms
Event Options.Process Person Profile Override ✅ 8ms
Event Options.Product Tour Id Override ✅ 9ms
Event Options.Unset Options Omitted ✅ 8ms
Event Options.Options Override In Batch ✅ 12ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties ✅ 8ms
Geoip And Historical Migration.Historical Migration Set In Body ✅ 9ms
Geoip And Historical Migration.Historical Migration Absent By Default ✅ 8ms

Capture_Ai Tests

⚠️ 4/5 tests passed, 1 failed

View Details
Test Status Duration
Routing.Capture Ai Posts To Ai Endpoint ✅ 8ms
Routing.Capture Does Not Reroute Ai Named Events ❌ 9ms
Identity.Capture Ai Event Has Uuid ✅ 8ms
Identity.Capture Ai Keeps Supplied Uuid ✅ 8ms
Timestamp Format.Non Utc Event Timestamp Is Converted To Utc ✅ 9ms

Failures

routing.capture_does_not_reroute_ai_named_events

Request 0 path '/i/v1/analytics/events' != expected '/batch'

Feature_Flags Tests

⚠️ 16/17 tests passed, 1 failed

View Details
Test Status Duration
Request Payload.Request With Person Properties Device Id ✅ 10ms
Request Payload.Flags Request Uses V2 Query Param ✅ 9ms
Request Payload.Flags Request Hits Flags Path Not Decide ✅ 10ms
Request Payload.Flags Request Omits Authorization Header ✅ 9ms
Request Payload.Token In Flags Body Matches Init ✅ 9ms
Request Payload.Groups Round Trip ✅ 9ms
Request Payload.Groups Default To Empty Object ✅ 10ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False ✅ 9ms
Request Payload.Disable Geoip Omitted Defaults To False ❌ 9ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key ✅ 9ms
Request Lifecycle.No Flags Request On Init Alone ✅ 4ms
Request Lifecycle.No Flags Request On Normal Capture ✅ 8ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests ✅ 13ms
Request Lifecycle.Mock Response Value Is Returned To Caller ✅ 9ms
Retry Behavior.Retries Flags On 502 ✅ 313ms
Retry Behavior.Retries Flags On 504 ✅ 313ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event ✅ 11ms

Failures

request_payload.disable_geoip_omitted_defaults_to_false

Expected geoip_disable=False, got True

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@dustinbyrne dustinbyrne

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@dustinbyrne