metadata: Changes to support gRPC Rust requirements

[arjan-bal]

Feature Request

gRPC Rust requires a Metadata API that is consistent with the gRPC over HTTP/2 protocol and capable of supporting true binary metadata (gRFC G1) in the future.

While evaluating the feasibility of reusing tonic's MetadataMap, I identified two major areas requiring modification:

1. Lazy encoding of binary values.
2. Stricter validation of header keys and values.

Motivation

In gRPC, binary metadata (keys ending with the -bin suffix) must be base64 encoded because the HTTP/2 specification discourages specific special characters in header values.

gRFC G1 describes an extension to the gRPC over HTTP/2 protocol where peers can use the HTTP/2 Settings frame to indicate support for raw (non-base64 encoded) metadata. This eliminates the CPU overhead of base64 encoding binary data. To implement this, the sender must inspect the peer's Settings frame to decide whether or not to encode the data.

Currently, MetadataMap encodes data immediately upon insertion into the map, incurring the CPU cost regardless of the peer's capabilities.

Proposal

Lazily encode binary data

To support gRFC G1 in the future, we must modify the internal structure of MetadataMap to store unencoded header values. This could remain an http::HeaderMap<UnencodedHeaderValue>, where UnencodedHeaderValue wraps the raw bytes.

The MetadataMap::into_headers and MetadataMap::from_headers methods would be updated to convert the internal http::HeaderMap<UnencodedHeaderValue> to/from an http::HeaderMap<http::HeaderValue>.

Because the encoded data would no longer be stored directly in the MetadataMap, APIs that provide a reference to encoded data must be removed. While they could theoretically be retained for Ascii values, keeping them for binary values would require returning unencoded data, which constitutes a larger behaviour change.

APIs to remove:

• MetadataValue::as_encoded_bytes
• impl<VE: ValueEncoding> AsRef<[u8]> for MetadataValue

Trait implementations to remove:
The following implementations expose the internal http::HeaderMap. These must be removed because the map will now hold unencoded values requiring conversion:

• impl AsMut for MetadataMap
• impl AsRef for MetadataMap

Stricter header validation

To ensure compliance with the gRPC over HTTP/2 protocol, the following changes regarding header validation are required:

1. ASCII Values: Disallow extended ASCII characters (128-255).
2. ASCII Values: Strip leading and trailing whitespace.
3. Key Validation: Header keys must only contain characters from the set 0-9 a-z _ - ..
4. Import Logic: In MetadataMap::from_headers, headers containing extended ASCII characters (128-255) or keys with invalid characters will be dropped/ignored.

Alternatives

I could implement a simplified MetadataMap for gRPC Rust that exposes an API similar to HashMap<String, Vec<String>> while hiding the internal data structure. For comparison, gRPC C++ and Go do not provide typed APIs for ASCII and binary metadata, instead operating directly on strings.

The following shows the proposed changes: arjan-bal#8

[arjan-bal]

cc @dfawley @LucioFranco

[LucioFranco]

@arjan-bal so for the first one we would need to create an RFC basically against the http repo to add support for an unencoded variant and then add support with h2?

and for the stricter header validation its not clear if that is something we need to do in MetadataValue or in HeaderValue?