[dotnet] Enable flag eval metrics tests

[sameerank]

Motivation

Prepare the .NET weblog for test_flag_eval_metrics.py tests to validate the new feature_flag.evaluations OTel metric implementation.

Related PRs:

• dd-trace-dotnet: [Feature Flags] Add flag evaluation metrics via OpenFeature hook dd-trace-dotnet#8367

Changes

Weblog (utils/build/docker/dotnet/weblog/):

• Fix JsonElement handling in FeatureFlagEvaluatorController.cs

  • ASP.NET Core's System.Text.Json deserializes object properties as JsonElement, not native types
  • Added helper methods to extract actual values from JsonElement before calling OpenFeature APIs
  • This prevents InvalidCastException that was occurring before OpenFeature hooks could fire

• Upgrade OpenFeature packages in app.csproj:

  • OpenFeature: 2.0.0 → 2.3.0 (required for FinallyAsync hook signature with FlagEvaluationDetails<T>)
  • Datadog.FeatureFlags.OpenFeature: Use wildcard version to pick up local/CI builds

Note: The tests/ffe/test_flag_eval_metrics.py tests remain marked as missing_feature in the manifest until dd-trace-dotnet v3.42.0 is released with flag eval metrics support.

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

utils/build/docker/dotnet/poc.Dockerfile                                @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core
utils/build/docker/dotnet/weblog/Controllers/FeatureFlagEvaluatorController.cs  @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core
utils/build/docker/dotnet/weblog/app.csproj                             @DataDog/apm-dotnet @DataDog/asm-dotnet @DataDog/system-tests-core

[sameerank]

Problem

The EvaluateRequest class has a DefaultValue property typed as object:

public class EvaluateRequest
{
    public object DefaultValue { get; set; }  // Can be bool, int, string, etc.
}

When ASP.NET Core's System.Text.Json deserializes JSON into an object property, it does not convert to native .NET types. Instead, it wraps the value in a JsonElement:

{ "defaultValue": true }     // Becomes JsonElement, not bool
{ "defaultValue": 42 }       // Becomes JsonElement, not int
{ "defaultValue": "hello" }  // Becomes JsonElement, not string

The original code assumed native types:

Convert.ToBoolean(request.DefaultValue)  // Throws! DefaultValue is JsonElement

Convert.ToBoolean() has no knowledge of JsonElement and throws InvalidCastException.

Why Previous Tests Didn't Fail

The original code had a catch block that swallowed all exceptions:

try
{
    value = ... Convert.ToBoolean(request.DefaultValue) ...
}
catch (Exception ex)
{
    value = request.DefaultValue;
    reason = "ERROR";
}

Previous tests (test_dynamic_evaluation.py, test_exposures.py) were asserting on:

• The evaluated flag value (returned by the endpoint)
• The reason field in the response

When InvalidCastException was thrown, the catch block returned reason = "ERROR" and value = request.DefaultValue. If a test expected an error case or didn't care about the reason, it would pass.

Why Flag Eval Metrics Tests Failed

The metrics tests need OpenFeature hooks to fire. The exception was thrown before calling the OpenFeature API:

1. Convert.ToBoolean(JsonElement) throws
2. The catch block catches it and returns "ERROR"
3. OpenFeature is never called → hooks never fire → no metrics emitted

The tests assert on metrics received by the OTLP intake, not on the HTTP response. No OpenFeature call = no hooks = no metrics = test failure.

The Fix

Check if the value is a JsonElement and extract the actual value:

private static bool GetDefaultValueAsBool(object defaultValue)
{
    if (defaultValue is JsonElement jsonElement)
    {
        return jsonElement.ValueKind switch
        {
            JsonValueKind.True => true,
            JsonValueKind.False => false,
            JsonValueKind.String => bool.Parse(jsonElement.GetString()),
            _ => false
        };
    }
    return Convert.ToBoolean(defaultValue);  // Fallback for non-JsonElement
}

[sameerank]

Leaving the test disabled for now because the changes in DataDog/dd-trace-dotnet#8367 haven't been published yet. Tests pass when run this locally: DataDog/dd-trace-dotnet#8367 (comment)

Copy-pasted from the referenced comment:

This is what I did (not 100% that this is the most efficient approach, but it works). I downloaded and extracted these .zip files:

and

And extracted datadog-dotnet-apm-3.42.0.arm64.tar.gz and copied the contents to
/Users/.../system-tests/binaries along with two *.nupkg files from nuget-packages:

And then did the following steps to run system tests locally

cd /Users/../system-tests
# Build the docker image
./build.sh --library dotnet --weblog-variant poc --images weblog
# Run tests
./run.sh FEATURE_FLAGGING_AND_EXPERIMENTATION --library dotnet -k "test_ffe_eval"
# Output shows that we are using the not-yet-published version 3.42.0 and tests are passing
=============================== test context ================================
Scenario: FEATURE_FLAGGING_AND_EXPERIMENTATION
Logs folder: ./logs_feature_flagging_and_experimentation
Starting containers...
Agent: 7.77.2
Backend: datad0g.com
Library: dotnet@3.42.0
Weblog variant: poc
Weblog system: Linux weblog 6.12.76-linuxkit #1 SMP Fri Mar  6 10:10:19 UTC 2026 aarch64 GNU/Linux

============================ test session starts ============================
collected 2427 items / 2410 deselected / 17 selected                        
-------------------------------- tests setup --------------------------------

tests/ffe/test_flag_eval_metrics.py .................

--------------------- Wait for library interface (40s) ----------------------
---------------------- Wait for agent interface (30s) -----------------------
---------------------- Wait for backend interface (0s) ----------------------

tests/ffe/test_flag_eval_metrics.py .................                 [100%]

- generated xml file: /Users/.../system-tests/logs_feature_flagging_and_experimentation/reportJunit.xml -
============== 17 passed, 2410 deselected in 175.48s (0:02:55) ==============

[sameerank]

Why run COPY binaries/ commands twice?

Stage 1: build-app (SDK image)

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build-app

# NEW: Copy binaries for NuGet package resolution
COPY binaries/ /binaries/
RUN if ls /binaries/*.nupkg 1> /dev/null 2>&1; then \
        dotnet nuget add source /binaries --name local-packages; \
    fi

# NuGet packages are resolved HERE
RUN dotnet restore
RUN dotnet publish

Purpose: Build the weblog application. NuGet packages (like Datadog.FeatureFlags.OpenFeature.*.nupkg) must be available as a NuGet source before dotnet restore runs.

Stage 2: runtime (ASP.NET image)

FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS runtime

# EXISTING: Copy binaries for tracer installation
COPY utils/build/docker/dotnet/install_ddtrace.sh binaries/ /binaries/
RUN /binaries/install_ddtrace.sh

Purpose: Install the Datadog tracer into the runtime image. The install_ddtrace.sh script uses:

• Native profiler: Datadog.Trace.ClrProfiler.Native.so
• Managed tracer DLLs: net6.0/Datadog.Trace.dll, etc.

[nccatoni]

LGTM (for @DataDog/system-tests-core) but you should get a review from someone more familiar with the weblog