System.Text.Json deserializes struct properties as null when reading from a stream - bug introduced in v9+

[jeremy-morren]

Description

For very specific JSON input when reading from a stream, a bug was introduced in .NET 9: if a property is a nullable struct, the value may be set to null even if the JSON input includes the value. If the property is a class, the deserialization succeeds.

Reproduction Steps

Please see below setup. The project with JSON input used is attached. Note that this test succeeds on .NET 8.

SystemTextJsonBug.zip

using System.Text.Json;
using Xunit;

// ReSharper disable ClassNeverInstantiated.Global
// ReSharper disable UnusedAutoPropertyAccessor.Global

namespace SystemTextJsonBug;

public class StjBugTests
{
    [Fact]
    public void WithStruct()
    {
        var filename = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Input.json");

        var bytes = File.ReadAllBytes(filename);

        var fromBytes = JsonSerializer.Deserialize<Model1Struct>(bytes);
        Assert.NotNull(fromBytes);
        Assert.NotNull(fromBytes.Scheduled[3].Job);
        Assert.NotNull(fromBytes.Scheduled[3].Job!.JobWeight.Remaining);
        Assert.True(fromBytes.Scheduled[3].Job!.JobWeight.Remaining!.Value.Total > 0);

        using var fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
        var fromStream = JsonSerializer.Deserialize<Model1Struct>(fs);
        Assert.NotNull(fromStream);
        Assert.NotNull(fromStream.Scheduled[3].Job);
        Assert.NotNull(fromStream.Scheduled[3].Job!.JobWeight.Remaining); // Fails on .NET 9+
        Assert.True(fromStream.Scheduled[3].Job!.JobWeight.Remaining!.Value.Total > 0);
    }

    [Fact]
    public void WithClass()
    {
        var filename = Path.Combine(AppDomain.CurrentDomain.BaseDirectory, "Input.json");

        var bytes = File.ReadAllBytes(filename);

        var fromBytes = JsonSerializer.Deserialize<Model1Class>(bytes);
        Assert.NotNull(fromBytes);
        Assert.NotNull(fromBytes.Scheduled[3].Job);
        Assert.NotNull(fromBytes.Scheduled[3].Job!.JobWeight.Remaining);
        Assert.True(fromBytes.Scheduled[3].Job!.JobWeight.Remaining!.Total > 0);

        using var fs = new FileStream(filename, FileMode.Open, FileAccess.Read);
        var fromStream = JsonSerializer.Deserialize<Model1Class>(fs);
        Assert.NotNull(fromStream);
        Assert.NotNull(fromStream.Scheduled[3].Job);
        Assert.NotNull(fromStream.Scheduled[3].Job!.JobWeight.Remaining);
        Assert.True(fromStream.Scheduled[3].Job!.JobWeight.Remaining!.Total > 0);
    }

    public class Model1Struct
    {
        public required Model2Struct[] Scheduled { get; init; }
    }

    public class Model2Struct
    {
        public required Model3Struct? Job { get; init; }
    }

    public class Model3Struct
    {
        public required Model4Struct JobWeight { get; init; }
    }

    public class Model4Struct
    {
        public required Model5Struct? Remaining { get; init; }
    }

    public readonly record struct Model5Struct
    {
        public required decimal Total { get; init; }
    }
    
    public class Model1Class
    {
        public required Model2Class[] Scheduled { get; init; }
    }

    public class Model2Class
    {
        public required Model3Class? Job { get; init; }
    }

    public class Model3Class
    {
        public required Model4Class JobWeight { get; init; }
    }

    public class Model4Class
    {
        public required Model5Class? Remaining { get; init; }
    }

    public record Model5Class
    {
        public required decimal Total { get; init; }
    }
}

Expected behavior

JSON input should successfully deserialize into property.

Actual behavior

If property is a nullable struct and when reading from a stream the property may be set to null.

Regression?

Yes, introduced on .NET 9.

Known Workarounds

Use class models instead of struct.

Configuration

No response

Other information

No response

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/area-system-text-json
See info in area-owners.md if you want to be subscribed.

[eiriktsarpalis]

Thanks for filing this, @jeremy-morren. I was able to independently reproduce and minimize the issue to a self-contained repro (see below). This is a confirmed regression from .NET 8 → .NET 9, still present in .NET 10.

The root cause appears to be in NullableConverter.OnTryRead — when stream deserialization hits a buffer boundary while parsing a nullable struct with unknown properties being skipped, the value is silently set to null.

Related: #110450 (same root cause area, manifests as JsonException instead of silent null).

Minimal Reproduction

using System.Text;
using System.Text.Json;

string json = """{"Value":{"Total":0,"a":null,"b":0}}""";
byte[] bytes = Encoding.UTF8.GetBytes(json);
var options = new JsonSerializerOptions { DefaultBufferSize = 32 };

var fromBytes = JsonSerializer.Deserialize<MyPoco>(bytes, options)!;
Console.WriteLine($"From bytes:  {fromBytes.Value}");
// MyStruct

using var stream = new MemoryStream(bytes);
var fromStream = JsonSerializer.Deserialize<MyPoco>(stream, options)!;
Console.WriteLine($"From stream: {fromStream.Value}");
// (null)

public class MyPoco { public MyStruct? Value { get; set; } }
public struct MyStruct { public int Total { get; set; } }