Enable EVP flagevaluation system tests for PHP by leoromanovsky · Pull Request #7187 · DataDog/system-tests

leoromanovsky · 2026-06-20T00:56:47Z

Motivation

Enable the server-side EVP flagevaluation system tests for PHP after the PHP SDK PR validated the shared contract, sidecar path, and local system-test path.

Changes

Final diff against main:

Carries the stacked shared flagevaluation contract update in tests/ffe/test_flag_eval_evp.py, including the singular /api/v2/flagevaluation path, no OpenFeature reason, omitted targeting_rule when no real rule metadata exists, evaluation-time first/last/count assertions, and degraded-bucket overflow coverage.
Changes tests/ffe/test_flag_eval_evp.py in manifests/php.yml from missing_feature (FFL-2446) to v1.21.0-dev.

Decisions

This PR is stacked on the shared flagevaluation contract-fix PR so the language-specific commit remains a single PHP manifest entry.
The matching SDK implementation is feat(openfeature): emit server-side EVP flagevaluation dd-trace-php#3984, backed by feat(datadog-ffe): server-side EVP flagevaluation payload + bincode-safe sidecar delivery libdatadog#2117 for the sidecar path.
The reviewer-facing diff is described against main, because the stack base only hides the shared contract file from GitHub's PR file list.
This PR remains draft until the SDK fanout review is complete.

Validation Evidence

git diff --check - PASS.
./tooling/bin/build-debug-artifact gnu-aarch64-8.0-nts /Users/leo.romanovsky/gsd-workspaces/flag-evaluations-cross-sdk/system-tests-clean-php/binaries from dd-trace-php - PASS; produced dd-library-php-1.21.0-aarch64-linux-gnu.tar.gz.
SYSTEM_TEST_BUILD_TIMEOUT=1800 ./build.sh --library php --weblog-variant apache-mod-8.0 from system-tests-clean-php - PASS on 2026-06-20.
TEST_LIBRARY=php WEBLOG_VARIANT=apache-mod-8.0 ./run.sh +l php FEATURE_FLAGGING_AND_EXPERIMENTATION -k "test_flag_eval_evp" - PASS on 2026-06-20, 8 passed, 2627 deselected in 60.08s with Library: php@1.21.0.
Degradation payload evidence from the passing system-test run: payloads=30, events=10195, total evaluation_count=12708; evp-degradation-flag emitted 10001 rows with total evaluation_count=12000, including one degraded row with evaluation_count=2000 and omitted targeting_key/context.
Real-backend dogfooding prefixes recorded in the PHP SDK PR:
- server-evp-e2e-php7-1781931066-user-
- server-evp-e2e-php8-openfeature-1781931066-user-

datadog-prod-us1-5 · 2026-06-20T00:57:27Z

Tests

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 52 Pipeline jobs failed

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0 1

🧪 1 Test failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[apache-mod-7.0] from system_tests_suite

AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 &#43;  where False = &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt;(&lt;function wait_for_evp_flagevaluation_event.&lt;locals&gt;.&lt;lambda&gt; at 0x7fe58b55a020&gt;, timeout=30)
 &#43;    where &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt; = AgentInterfaceValidator(&#39;agent&#39;).wait_for
 &#43;      where AgentInterfaceValidator(&#39;agent&#39;) = interfaces.agent

self = &lt;tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7fe5bc999dc0&gt;

    def test_ffe_evp_flagevaluation_basic(self) -&gt; None:
        assert self.r.status_code == 200, f&#34;Flag evaluation failed: {self.r.text}&#34;
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1

🧪 1 Test failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[apache-mod-7.0-zts] from system_tests_suite

AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 &#43;  where False = &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt;(&lt;function wait_for_evp_flagevaluation_event.&lt;locals&gt;.&lt;lambda&gt; at 0x7feebfbf91c0&gt;, timeout=30)
 &#43;    where &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt; = AgentInterfaceValidator(&#39;agent&#39;).wait_for
 &#43;      where AgentInterfaceValidator(&#39;agent&#39;) = interfaces.agent

self = &lt;tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7feedfb0e210&gt;

    def test_ffe_evp_flagevaluation_basic(self) -&gt; None:
        assert self.r.status_code == 200, f&#34;Flag evaluation failed: {self.r.text}&#34;
...

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.1 1

🧪 1 Test failed

tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic.test_ffe_evp_flagevaluation_basic[apache-mod-7.1] from system_tests_suite

AssertionError: Timed out waiting for EVP flagevaluation event for flag evp-basic-flag
assert False
 &#43;  where False = &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt;(&lt;function wait_for_evp_flagevaluation_event.&lt;locals&gt;.&lt;lambda&gt; at 0x7fe90aa845e0&gt;, timeout=30)
 &#43;    where &lt;bound method ProxyBasedInterfaceValidator.wait_for of AgentInterfaceValidator(&#39;agent&#39;)&gt; = AgentInterfaceValidator(&#39;agent&#39;).wait_for
 &#43;      where AgentInterfaceValidator(&#39;agent&#39;) = interfaces.agent

self = &lt;tests.ffe.test_flag_eval_evp.Test_FFE_EVP_Flagevaluation_Basic object at 0x7fe938100890&gt;

    def test_ffe_evp_flagevaluation_basic(self) -&gt; None:
        assert self.r.status_code == 200, f&#34;Flag evaluation failed: {self.r.text}&#34;
...

View all 52 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f7fb6d9 | Docs | Datadog PR Page | Give us feedback!}

datadog-system-tests-org · 2026-06-20T00:57:40Z

⚠️ Warnings

🚦 52 Pipeline jobs failed

Codeowners | Report codeowners

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0 1

Testing the test | System Tests (php, dev) / End-to-end #1 / apache-mod-7.0-zts 1

View all 52 failed jobs.

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f7fb6d9 | Docs | Give us feedback!}

…afe sidecar delivery (#2117) ## Motivation PHP relies on libdatadog and the sidecar for server-side flag-evaluation delivery, so the shared sidecar path must preserve the same worker-facing EVP contract as the direct SDK implementations. This contribution gives PHP a reusable, backend-verifiable flagevaluation transport without adding a one-off EVP writer in the PHP extension, which keeps cross-SDK adoption smoother for SDK owners and APM review. ## Changes - Keeps the sidecar action enum bincode-compatible by appending `FfeFlagEvaluationBatch` after existing variants. - Keeps flagevaluation IPC structs bincode-safe; outbound placeholder stripping happens only in the sidecar flusher before the EVP POST. - Shares the common Agent EVP proxy endpoint/header/request path between FFE exposures and flagevaluation delivery. - Carries `context.evaluation` over IPC as a JSON-object string and re-expands it to an object for outbound EVP JSON. - Preserves valid context values such as `false`, `""`, `{}`, and `[]` in user context. - Adds the flagevaluation C ABI path without a `reason` argument. - Supports optional real `targeting_rule.key`. - Retries reliable enqueue using retained encoded bytes so reconnect retry does not consume the payload. - Coalesces PHP flagevaluation batches in the sidecar by schema-visible dimensions and folds overflow into degraded buckets. - Splits sidecar EVP POST bodies by encoded uncompressed JSON bytes under the 5 MiB EVP limit, keeping the existing 512-event chunking as a secondary guard. - Degrades an oversized full-tier row by omitting `targeting_key` and `context`; drops and logs only if the degraded row still cannot fit. ## Decisions - OpenFeature `reason` is not part of the native ABI, sidecar event, outbound EVP payload, or aggregation contract. - Bincode wire compatibility is part of the PHP product path because PHP reaches EVP through worker-to-sidecar IPC before HTTP. - FFE exposures and flagevaluation share Agent EVP proxy transport mechanics; schema-specific payload shaping remains in each flusher because exposure deduplication and flagevaluation coalescing have different contracts. - The flagevaluation sidecar flusher owns worker-facing JSON cleanup so shared wire structs remain plain enough for positional bincode serialization. - Degraded events omit targeting key and context while retaining visible flag, variant, allocation, error, runtime-default, and targeting-rule fields. - Payload splitting happens after sidecar JSON cleanup because that is where the final uncompressed POST body exists; count-based chunking alone is not sufficient for the EVP 5 MiB limit. - PHP consumes this flagevaluation endpoint through the C ABI; the companion PHP draft points its libdatadog submodule at this PR head. ```mermaid flowchart TD A[sidecar receives/coalesces batches] --> B[clean and encode event JSON] B --> C{candidate POST <= 5 MiB?} C -- yes --> D[POST through Agent EVP proxy] C -- no --> E{single full row can degrade?} E -- yes --> F[omit targeting_key and context] F --> B E -- no --> G[drop and log/count] ``` ## Validation Evidence ### Dogfooding App - Companion PHP draft: DataDog/dd-trace-php#3984 - `ffe-dogfooding` `app-php7` and `app-php8-openfeature` were run with local PHP artifacts using this libdatadog sidecar path. - Dogfooding ran with the real-backend EVP override: Agent `DD_EVP_PROXY_CONFIG_ADDITIONAL_ENDPOINTS={}`, `DD_SKIP_SSL_VALIDATION=false`, and `DD_REMOTE_CONFIGURATION_NO_TLS_VALIDATION=false`. - Sent 12 identical evaluations per targeting key for `ffe-dogfooding-string-flag`: - `libdd-php7-batch-20260623T020845Z-alpha` - `libdd-php7-batch-20260623T020845Z-bravo` - `libdd-php8of-batch-20260623T020845Z-alpha` - `libdd-php8of-batch-20260623T020845Z-bravo` - App-side responses returned `variant_2`. ### System Tests - Companion draft PR: DataDog/system-tests#7187 ### Staging End-To-End - Retriever staging query used `--datacenter us1.staging.dog --customer-auth=skip` against the `flagevaluation` track for the exact targeting keys above. - Retriever returned one backend row per targeting key, each with `flag.key=ffe-dogfooding-string-flag`, `variant.key=variant_2`, `allocation.key=allocation-override-392dd7c149f8`, and `evaluation_count=12`: - `libdd-php7-batch-20260623T020845Z-alpha`: `first_evaluation=1782180525878`, `last_evaluation=1782180525985`, `timestamp=1782180525985`, `evaluation_count=12` - `libdd-php7-batch-20260623T020845Z-bravo`: `first_evaluation=1782180525997`, `last_evaluation=1782180526103`, `timestamp=1782180526103`, `evaluation_count=12` - `libdd-php8of-batch-20260623T020845Z-alpha`: `first_evaluation=1782180526245`, `last_evaluation=1782180526344`, `timestamp=1782180526344`, `evaluation_count=12` - `libdd-php8of-batch-20260623T020845Z-bravo`: `first_evaluation=1782180526356`, `last_evaluation=1782180526451`, `timestamp=1782180526451`, `evaluation_count=12` --------- Co-authored-by: pr-shepherd-6ad11f[bot] <305617008+pr-shepherd-6ad11f[bot]@users.noreply.github.com> Co-authored-by: vickie.fridge <vickie.fridge@datadoghq.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

leoromanovsky added 2 commits June 19, 2026 20:38

Fix EVP flagevaluation system test contract

c0b4263

Enable EVP flagevaluation system test for PHP

f7fb6d9

Base automatically changed from leo.romanovsky/ffe-evp-flagevaluation-contract-fix to main June 22, 2026 19:40

This was referenced Jun 22, 2026

feat(openfeature): emit server-side EVP flagevaluation DataDog/dd-trace-php#3984

Draft

feat(datadog-ffe): server-side EVP flagevaluation payload + bincode-safe sidecar delivery DataDog/libdatadog#2117

Merged

Navigation Menu

Uh oh!

Enable EVP flagevaluation system tests for PHP - #7187

Enable EVP flagevaluation system tests for PHP#7187
leoromanovsky wants to merge 2 commits into
mainfrom
leo.romanovsky/ffe-evp-flagevaluation-enable-php-clean

leoromanovsky commented Jun 20, 2026 •

edited

Loading

Uh oh!

datadog-prod-us1-5 Bot commented Jun 20, 2026 •

edited by datadog-official Bot

Loading

Uh oh!

datadog-system-tests-org Bot commented Jun 20, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

leoromanovsky commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Changes

Decisions

Validation Evidence

Uh oh!

datadog-prod-us1-5 Bot commented Jun 20, 2026 • edited by datadog-official Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

ℹ️ Info

Uh oh!

datadog-system-tests-org Bot commented Jun 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ Warnings

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

leoromanovsky commented Jun 20, 2026 •

edited

Loading

datadog-prod-us1-5 Bot commented Jun 20, 2026 •

edited by datadog-official Bot

Loading

datadog-system-tests-org Bot commented Jun 20, 2026 •

edited

Loading