fix(flags): parse locally-evaluated flag payloads in evaluate_flags() (sdk-specs local-feature-flag-evaluator)

[posthog]

💡 Motivation and Context

Compliance gap against the cross-SDK contracts in PostHog/sdk-specs.

openspec/specs/local-feature-flag-evaluator/spec.md — Behavior #8 "Resolve payload from the chosen value", Scenario "Evaluator resolves payload from the matched value":

AND local feature flag definitions include a multivariate flag "checkout" with variant "blue" and payload: | copy | new |
WHEN local feature flag "checkout" is evaluated for distinct id "user-123"
THEN the local evaluation payload should include: | copy | new |

openspec/specs/get-feature-flag-payload/spec.md says the same thing about the read path:

Parse serialized payloads if necessary. Some SDKs store payloads as JSON strings in cache and parse them on read before returning them.

What was out of compliance. In Client.evaluate_flags(), the remote branch JSON-decodes metadata.payload before building the _EvaluatedFlagRecord, but the local branch stored the value straight from the flag definition — where payloads live as JSON strings. So the payload type of the same flag depended on where it happened to resolve:

flags = posthog.evaluate_flags("user-123")
flags.get_flag_payload("checkout")
# poller resolved it locally  -> '{"copy": "new"}'   (str)
# fell back to /flags         -> {"copy": "new"}     (dict)

The rest of the SDK already treats "payload" as the decoded value — FeatureFlagResult.from_value_and_payload / from_flag_details both json.loads, so get_feature_flag_result(...).payload returns a dict for the very same locally-evaluated flag that evaluate_flags(...).get_flag_payload(...) returned a string for. The same raw string also leaked into $feature_flag_payload on $feature_flag_called events for locally-evaluated flags, while the remote path sent the decoded value (the existing test at posthog/test/test_feature_flag_called_minimization.py notes "The evaluate_flags path JSON-parses the payload before attaching it" — which held only for remote).

How this fixes it. Both branches now route through one _parse_flag_payload() helper, so a decoded payload is what callers see regardless of evaluation source. The helper preserves the remote branch's existing semantics exactly: an empty string becomes None, a string that isn't valid JSON is passed through unchanged, and non-string values (a number or an already-structured payload) are untouched.

Behavior change / compatibility risk. evaluate_flags(...).get_flag_payload(key) and $feature_flag_payload now return the decoded payload for locally-evaluated flags instead of a JSON string. Anyone who called json.loads() on that result will need to drop it — but only for flags that resolved locally, since the same code already received a decoded value on the remote path, so no caller could have relied on the string shape reliably. No public API, signature, config, or event-name changes.

Deliberately scoped to the evaluate_flags() snapshot. get_all_flags_and_payloads() returns JSON strings on both its local and remote paths, so it is internally consistent today; changing it is a wider, unambiguously breaking change and belongs in its own discussion.

💚 How did you test it?

Added TestEvaluateFlagsLocalPayloads in posthog/test/test_evaluate_flags.py, covering local evaluation of a multivariate and a boolean flag (decoded payload, no /flags request), a non-JSON payload string (passed through), and the $feature_flag_payload property on $feature_flag_called.

Verified the two payload assertions fail without the fix (AssertionError: '{"copy": "new"}' != {'copy': 'new'}) and pass with it. Also ran test_evaluate_flags.py, test_feature_flags.py, test_feature_flag.py, test_feature_flag_result.py, test_feature_flag_called_minimization.py, test_feature_flag_has_experiment.py, test_types.py, test_client.py, test_module.py, test_contexts.py and posthog/test/features — all passing. ruff format --check, ruff check, mypy, and check_public_api.py are clean (no public API surface change).

📝 Checklist

• I reviewed the submitted code.
• I added tests to verify the changes.
• I updated the docs if needed.
• No breaking change or entry added to the changelog.

If releasing new changes

• Ran sampo add to generate a changeset file

🤖 Agent context

Autonomy: Fully autonomous

Opened by the scheduled "SDK Spec Compliance Enforcer" loop, which audits posthog-python against the contracts in PostHog/sdk-specs and files one focused fix per run. This run checked the server-SDK-applicable requirements across ~40 specs (capture/exception, batching + HTTP/retry, flag getters, flag internals, identity, logs/traces/tracing-headers) using Claude Code with read-only search plus targeted runs of the SDK against mocked transports.

This finding was chosen over the other candidates because it is narrow, verifiable against two spec files, and internally corroborated: the SDK's own FeatureFlagResult and the remote branch of the same function already decode payloads, so this is the recommended non-deprecated API regressing against a convention the rest of the codebase keeps. Candidates deliberately not filed: the group-flag-without-group-context "conclusive False" behavior (the code comment marks it intentional and it is shared with other server SDKs — needs a cross-SDK decision), get_all_flags_and_payloads() payload strings (wider breaking change), and missing input validation on alias() / group_identify() (real but separate; would drop events users send today).

Agent-authored — please review rather than rubber-stamp, especially the compatibility call on returning decoded payloads.

---

Created with PostHog Code

[github-actions]

posthog-python Compliance Report

Date: 2026-08-05 11:24:02 UTC
Duration: 338930ms

✅ All Tests Passed!

111/111 tests passed

---

Capture_V1 Tests

✅ 94/94 tests passed

View Details

Test	Status	Duration
Endpoint And Method.Targets V1 Endpoint	✅	517ms
Endpoint And Method.Does Not Use Legacy Endpoints	✅	1009ms
Required Headers.Has Authorization Bearer Header	✅	1010ms
Required Headers.Has Content Type Json	✅	1010ms
Required Headers.Has Posthog Sdk Info Format	✅	1010ms
Required Headers.Has Posthog Attempt Header	✅	1010ms
Required Headers.Has Posthog Request Id	✅	1009ms
Required Headers.Has Posthog Request Timestamp	✅	1009ms
Required Headers.Has User Agent	✅	1010ms
Body Format.Body Has Created At And Batch	✅	1009ms
Body Format.No Api Key In Body	✅	1010ms
Body Format.No Sent At In Body	✅	1010ms
Event Format.Event Has Required Root Fields	✅	1011ms
Event Format.Event Uuid Is Valid	✅	1009ms
Event Format.Event Timestamp Is Rfc3339	✅	1010ms
Event Format.Distinct Id Is String	✅	1010ms
Event Format.Distinct Id At Root Not Properties	✅	1011ms
Event Format.Custom Properties Preserved	✅	1011ms
Event Format.Set Properties Preserved	✅	1009ms
Event Format.Set Once Properties Preserved	✅	1009ms
Event Format.Groups Properties Preserved	✅	1011ms
Event Format.Sdk Generates Uuid If Not Provided	✅	1009ms
Event Format.Event Has Required Root Fields Batch	✅	1014ms
Event Format.Event Uuid Is Valid Batch	✅	1013ms
Event Format.Event Timestamp Is Rfc3339 Batch	✅	1014ms
Event Format.Distinct Id Is String Batch	✅	1013ms
Event Format.Distinct Id At Root Not Properties Batch	✅	1014ms
Event Format.Custom Properties Preserved Batch	✅	1015ms
Event Format.Set Properties Preserved Batch	✅	1013ms
Event Format.Set Once Properties Preserved Batch	✅	1013ms
Event Format.Groups Properties Preserved Batch	✅	1014ms
Event Format.Sdk Generates Uuid If Not Provided Batch	✅	1014ms
Batch Behavior.Multiple Events In Single Batch	✅	1508ms
Batch Behavior.Batch Envelope Smoke	✅	1017ms
Batch Behavior.Flush With No Events Sends Nothing	✅	1005ms
Batch Behavior.Flush At Triggers Batch	✅	1510ms
Batch Behavior.Created At Reflects Batch Creation Time	✅	1012ms
Deduplication.Generates Unique Uuids	✅	1507ms
Deduplication.Different Events Same Content Different Uuids	✅	1508ms
Deduplication.Preserves Uuid On Retry	✅	7516ms
Deduplication.Preserves Timestamp On Retry	✅	7510ms
Deduplication.Preserves Uuid And Timestamp On Batch Retry	✅	7515ms
Deduplication.No Duplicate Events In Batch	✅	1503ms
Header Behavior On Retry.Attempt Header Starts At One	✅	1009ms
Header Behavior On Retry.Attempt Header Increments On Retry	✅	14524ms
Header Behavior On Retry.Request Id Preserved On Retry	✅	7504ms
Header Behavior On Retry.Different Requests Have Different Request Ids	✅	3520ms
Header Behavior On Retry.Request Timestamp Changes On Retry	✅	7511ms
Response Format Validation.Success Response Has Uuid Keyed Results	✅	1011ms
Response Format Validation.Success Response Has Ok For Each Event	✅	1507ms
Response Format Validation.Success No Retry After When All Ok	✅	1507ms
Response Format Validation.Success Retry After Present When Retry Events	✅	2511ms
Response Format Validation.Success No Retry After When Drop Only	✅	1507ms
Response Format Validation.Response Echoes Request Id	✅	1010ms
Retry Behavior.Retries On 408	✅	7516ms
Retry Behavior.Retries On 500	✅	7511ms
Retry Behavior.Retries On 503	✅	9514ms
Retry Behavior.Retries On 504	✅	7513ms
Retry Behavior.Retryable Errors Have Retry After	✅	4511ms
Retry Behavior.Respects Retry After On Retryable Error	✅	12510ms
Retry Behavior.Does Not Retry On 400	✅	3510ms
Retry Behavior.Does Not Retry On 401	✅	3508ms
Retry Behavior.Does Not Retry On 402	✅	3508ms
Retry Behavior.Does Not Retry On 413	✅	3509ms
Retry Behavior.Does Not Retry On 415	✅	3506ms
Retry Behavior.Non Retryable Errors Have No Retry After	✅	3509ms
Retry Behavior.Implements Backoff	✅	23522ms
Retry Behavior.Max Retries Respected	✅	23531ms
Partial Batch Handling.Handles 200 Full Success	✅	3000ms
Partial Batch Handling.Handles 200 With All Ok	✅	4508ms
Partial Batch Handling.Does Not Retry Dropped Events	✅	4508ms
Partial Batch Handling.Does Not Retry Limited Events	✅	4508ms
Partial Batch Handling.Prunes Ok Events On Partial Retry	✅	7513ms
Partial Batch Handling.Prunes Dropped Events On Partial Retry	✅	7513ms
Partial Batch Handling.Retries Only Retry Events From Partial	✅	7511ms
Partial Batch Handling.Partial Retry Preserves Uuids	✅	7512ms
Partial Batch Handling.Partial Retry Attempt Header Increments	✅	7511ms
Partial Batch Handling.Partial Retry Request Id Preserved	✅	7507ms
Partial Batch Handling.Respects Retry After On Partial	✅	9516ms
Partial Batch Handling.Unknown Result Treated As Terminal	✅	4505ms
Partial Batch Handling.Mixed Ok Drop Limited No Retry	✅	4510ms
Compression.Sends Gzip Content Encoding	✅	1006ms
Compression.No Content Encoding When Disabled	✅	1009ms
Compression.Compressed Body Is Decompressible	✅	1008ms
Error Handling.Does Not Retry On Unknown 4Xx	✅	3510ms
Event Options.Cookieless Mode Override	✅	1007ms
Event Options.Disable Skew Correction Override	✅	1008ms
Event Options.Process Person Profile Override	✅	1009ms
Event Options.Product Tour Id Override	✅	1009ms
Event Options.Unset Options Omitted	✅	1008ms
Event Options.Options Override In Batch	✅	1012ms
Geoip And Historical Migration.Geoip Disable Injected Into Properties	✅	1008ms
Geoip And Historical Migration.Historical Migration Set In Body	✅	1009ms
Geoip And Historical Migration.Historical Migration Absent By Default	✅	1008ms

Feature_Flags Tests

✅ 17/17 tests passed

View Details

Test	Status	Duration
Request Payload.Request With Person Properties Device Id	✅	1007ms
Request Payload.Flags Request Uses V2 Query Param	✅	1006ms
Request Payload.Flags Request Hits Flags Path Not Decide	✅	1006ms
Request Payload.Flags Request Omits Authorization Header	✅	1007ms
Request Payload.Token In Flags Body Matches Init	✅	1006ms
Request Payload.Groups Round Trip	✅	1007ms
Request Payload.Groups Default To Empty Object	✅	1006ms
Request Payload.Disable Geoip False Propagates As Geoip Disable False	✅	1007ms
Request Payload.Disable Geoip Omitted Defaults To False	✅	1006ms
Request Payload.Flag Keys To Evaluate Contains Only Requested Key	✅	1007ms
Request Lifecycle.No Flags Request On Init Alone	✅	502ms
Request Lifecycle.No Flags Request On Normal Capture	✅	1507ms
Request Lifecycle.Two Flag Calls Produce Two Remote Requests	✅	1011ms
Request Lifecycle.Mock Response Value Is Returned To Caller	✅	1002ms
Retry Behavior.Retries Flags On 502	✅	1007ms
Retry Behavior.Retries Flags On 504	✅	1006ms
Side Effect Events.Get Feature Flag Captures Feature Flag Called Event	✅	1509ms