Response Header Deserialization

[joheredi]

Executive Summary

This design document outlines a feature enhancement to the TypeScript SDK emitter that enables automatic deserialization of HTTP response headers into strongly-typed model interfaces. This aligns SDK behavior with HTTP semantics where response headers carry important metadata alongside the response body, and provides developers with a unified, type-safe API for accessing all response data.

Problem Statement

Current State

In HTTP, responses consist of two distinct parts:

• Headers: Metadata about the response (e.g., x-user-id, content-type, etag)
• Body: The main response payload

TypeScript SDK-generated clients currently deserialize only the response body into a model type, leaving headers as untyped data available only through raw result.headers access. This creates several issues:

1. Type Safety Loss: Header values are strings that developers must manually parse and type-cast
2. Incomplete Models: Response models don't reflect the complete response contract
3. Developer Friction: Developers must know which headers exist and manually check result.headers for critical values
4. API Inconsistency: Response body is fully typed, but important header values are not

User Perspective

When a service contract specifies that a response includes both body data AND header metadata (e.g., a versioning header, request ID, or custom business data):

model UserResponse {
  id: string;
  name: string;
  
  @header("x-user-version")
  version: string;
  
  @header("x-request-id")
  requestId: string;
}

op getUser(): UserResponse;

Users expect to access all response data through a single, typed interface:

const user = await getUser();
console.log(user.id);           // ✓ Easy, typed
console.log(user.name);         // ✓ Easy, typed
console.log(user.version);      // Currently: ✗ Not available directly
console.log(user.requestId);    // Currently: ✗ Not available directly

Desired behavior: Headers should be seamlessly integrated into the response model, just like body properties.

Solution Overview

Design Approach

Transform the deserialization pipeline to:

1. Make header properties part of the response model interface
2. Mark header properties as optional (since headers may not always be present)
3. During deserialization, extract both body and header values and merge them into the response object
4. Maintain backward compatibility with existing clients that don't use headers

Detailed Design

1. Public API Changes: Response Model Interfaces

Change 1.1: Include Header Properties in Model Interfaces

Before:

export interface UserResponse {
  id: string;
  name: string;
}

After:

export interface UserResponse {
  id: string;
  name: string;
  version?: string;        // From @header("x-user-version")
  requestId?: string;      // From @header("x-request-id")
}

Rationale:

• Headers are part of the HTTP response contract and should be visible in the model
• Optional modifier (?) reflects HTTP reality: headers may not always be present
• Developers get full type information without breaking existing code that doesn't use headers

Change 1.2: Header Properties Always Optional

Header properties are always marked as optional (with ? modifier) regardless of their TypeSpec definition because:

1. HTTP Semantics: HTTP headers are fundamentally optional; clients must handle cases where headers are absent
2. Network Reliability: Response headers can be lost or stripped by intermediaries (proxies, gateways, CDNs)
3. Defensive Programming: Even if a service promises to send a header, defensive clients should handle its absence
4. API Evolution: Services may remove or conditionally omit headers in future versions

export interface UserResponse {
  // Body properties keep their original optionality
  id: string;           // Required
  email?: string;       // Optional
  
  // Header properties are always optional
  version?: string;     // Always optional, even if TypeSpec declares it required
  requestId?: string;   // Always optional, even if TypeSpec declares it required
}

2. Deserializer Changes: Response Object Construction

Change 2.1: Deserializer Signature Enhancement

Before:

export function userResponseDeserializer(item: any): UserResponse {
  return {
    id: item["id"],
    name: item["name"]
  };
}

After:

export function userResponseDeserializer(item: any, headers?: any): UserResponse {
  return {
    id: item["id"],
    name: item["name"],
    version: headers?.["x-user-version"],
    requestId: headers?.["x-request-id"]
  };
}

Key Points:

• headers parameter is optional (has ? modifier), maintaining backward compatibility
• Uses optional chaining (?.) when accessing headers to safely handle undefined values
• Header values are read using their wire names (e.g., lowercase with hyphens: "x-user-version")
• Straightforward object property assignment; no complex transformation logic

Change 2.2: Selective Header Parameter Passing

To avoid unnecessary parameter passing and maintain clean signatures:

Only pass headers when the model has header properties:

// Model WITH header properties: pass headers
return userResponseDeserializer(result.body, result.headers);

// Model WITHOUT header properties: don't pass headers
return basicMetadataDeserializer(result.body);

Benefits:

• Generated code remains clean and readable
• Minimal overhead for models that don't use headers
• Type-safe: TypeScript validates correct number of arguments

Change 2.3: Handling Inheritance

For models with inheritance, header properties from base models are also extracted:

model Pet {
  name: string;
  
  @header("x-pet-version")
  version: string;
}

model Cat extends Pet {
  meow: boolean;
  
  @header("x-cat-color")
  color?: string;
}

// Deserializer handles headers from both Cat and Pet
export function catDeserializer(item: any, headers?: any): Cat {
  return {
    // Pet properties
    name: item["name"],
    version: headers?.["x-pet-version"],
    // Cat properties
    meow: item["meow"],
    color: headers?.["x-cat-color"]
  };
}

3. Operation Deserialization Integration

Operations that return models with headers automatically pass headers during deserialization:

export async function _getUserDeserialize(
  result: PathUncheckedResponse
): Promise<UserResponse> {
  const expectedStatuses = ["200"];
  if (!expectedStatuses.includes(result.status)) {
    throw createRestError(result);
  }
  
  // Automatically includes headers in deserialization
  return userResponseDeserializer(result.body, result.headers);
}

export async function getUser(
  context: Client,
  options: GetUserOptionalParams = { requestOptions: {} }
): Promise<UserResponse> {
  const result = await _getUserSend(context, options);
  return _getUserDeserialize(result);
}

HTTP Alignment

This design aligns with HTTP specifications and best practices:

RFC 7230/7231: HTTP Semantics and Header Fields

• Recognizes headers as a first-class part of response semantics
• Treats headers as metadata that may be important to the application

REST API Design Principles

• Semantic Completeness: Response objects now contain complete response semantics
• Defensive Programming: Always-optional headers follow HTTP robustness principle
• Layered Abstraction: Hides HTTP plumbing (headers vs. body distinction) from SDK users

Backward Compatibility

✅ Fully Backward Compatible

• Existing code that doesn't access header properties continues to work without changes
• New header properties are optional and default to undefined
• Deserializers can be called with or without the headers parameter
• Models that don't have header properties are unaffected

Developer Experience Benefits

1. Type Safety

// Before: Manual string access, no type checking
const version = result.headers?.["x-user-version"];  // type: string | undefined

// After: Full type checking through model interface
const user = await getUser();
const version = user.version;  // type: string | undefined, caught by IDE/linter

2. Unified Response Access

// Before: Mixed access patterns
const user = {
  id: deserializedObject.id,
  version: rawHeaders["x-user-version"]
};

// After: Consistent model-based access
const user = await getUser();  // All properties in one place

3. IntelliSense Support

// IDE provides full autocomplete for all response properties
const user = await getUser();
user.  // ← IDE shows: id, name, version, requestId

4. Documentation

Types serve as inline documentation:

// Type definition shows all available response data
interface UserResponse {
  id: string;           // User ID
  name: string;         // User display name
  version?: string;     // API version from x-user-version header
  requestId?: string;   // Request tracking ID from x-request-id header
}

Implementation Details

Scope of Changes

• Model Interface Generation (emitModels.ts): Include header properties, mark as optional
• Deserializer Generation (buildDeserializerFunction.ts): Add optional headers parameter, extract header values
• Operation Deserialization (operationHelpers.ts): Conditionally pass headers based on model composition

Detection Logic

• Uses @header TypeSpec decorator to identify header properties
• Scans model and all ancestor models for header properties
• Only passes headers parameter when model contains header properties

[timovv]

Looks good! A couple of questions but nothing major

[timovv]

Might be missing something important here -- but what makes headers different from properties in the request body to motivate making them optional in the response model? I get that headers might not always be present if the service is not following the spec, but the same argument could be made for any field on the request body too, and those fields are still generated as required.

[timovv]

Looks like headers comes straight from the PathUncheckedResponse when _getUserDeserialize calls this. Will it ever be undefined?

[timovv]

Nit, could TCGC help here? I see there is a SdkServiceResponseHeader type that looks promising but couldn't tell at a glance whether we have access to that info here

[MaryGao]

As I mentioned, TCGC isn't fully ready to support this case. So currently the runtime in TCGC has some limitation that we need to touch the raw data I think.

[maorleger]

Not too well-versed with the implementation details; however, I did spot one thing I had a question about. Overall though, the approach seems ok to me.

It aligns with our guidelines:

The logical entity is a protocol neutral representation of a response. For HTTP, the logical entity may combine data from headers, body and the status line. A common example is exposing an ETag header as a property on the logical entity in addition to any deserialized content from the body.

and with Python's implementation as far as I can tell.

Do you know how it'll deserialize x-ms headers? Just curious

[maorleger]

would this need changing as well? Wouldn't XML deserializer be called without the header parameter then?

[qiaozha]

@MaryGao are you following up with this? Is this the accepted approach for response model related breakings?

[qiaozha]

Could you hold this PR until we get @MaryGao 's confirm?

/cc @lirenhe we need to prioritize this response model type related breaking discussion as soon as possible.

[MaryGao]

Could you hold this PR until we get @MaryGao 's confirm?

/cc @lirenhe we need to prioritize this response model type related breaking discussion as soon as possible.

We have some offline discussions and here are the agreements with Maor and Jeff. In short:

• No generation in Modular FooResponse = Foo & FooResponseHeaders
• Treat non-model as special case to mitigate breaking
  • non-model type without headers e.g {body: string; }
  • non-model type with headers e.g {body: string; fooHeader: string; barHeader: string; }
• To include headers in model, Jose's PR would adopt bodyRoot nested with header ideas to mitigate storage breaking.
  • To un-block storage we plan to have this design firstly, we could review this again once any conclusion reached out cross team in TCGC.

Overall the decision looks good to me and I will review this PR for the adoption and verify the impact in our side.

[MaryGao]

Any reason on we specially handle header-only responses?

This would introduce some changes in ARM SDKs where the return type is void.

[MaryGao]

As I mentioned this is the case I have concerns. I wonder if we have real case in storeage yet. If no, I prefer we figure out how to express following two scenarios in TypeSpec side first.

• headers are included in response return type
• headers are not included in return type by intention

Currently we would include all headers in responses and this is common that ARM SDKs would introduce new changes especially DELETE with void return type.

If we have changed the design in future, this would be a breaking for ARM libraries. Or maybe we could only enable this for DPG, right now?

[MaryGao]

Have we implemented other cases in gist?

I may wonder if the bodyRoot case is supported: https://gist.github.com/joheredi/d1e310b6d1464ae89fcaefb00601693d#file-3_bodyroot_decorator-md.

model User {
  name: string;
  email: string;

  @header("x-user-id")
  userId?: string;

  @header
  @encode("rfc7231")
  createdAt?: utcDateTime;
}

op getUser(): { @bodyRoot user: User} & { @header fooHeader: string};

[MaryGao]

I looked at the recent storage spec with below example in which the relevant headers would NOT generate in responses(which may not meet our expectation). But the behavior meets my expectation around this PR. My assumption for this PR would be

• if these headers are explicilty added into bodyRoot response type, it will be included in our response type;
• if these headers are not added into bodyRoot or body response type, it will be not be there

Please note some cases, TypeSpec HTTP lib would have an implicit body resolution: https://typespec.io/docs/libraries/http/operations/#implicit-body-resolution.

getAccessPolicy is StorageOperation<
    {
      ...TimeoutParameter;
      ...LeaseIdOptionalParameter;
    },
    {
      /** Signed identifiers */
      @body body: SignedIdentifiers;

      ...BlobPublicAccess;
      ...EtagResponseHeader;
      ...LastModifiedResponseHeader;
    }
  >;

[MaryGao]

Overall the PR looks good to me and I left some comments for clarification.

[MaryGao]

Have we handled request headers also?

model User {
  id: string;
  name: string;
  
  @header("x-user-version")
  version: string;
  
  @header("x-request-id")
  requestId: string;
}

op getUser(user: User ): User;