Add pre-TLS processing hook for PROXY protocol support and expose methods for HTTP2 shutdown#799
Add pre-TLS processing hook for PROXY protocol support and expose methods for HTTP2 shutdown#799jaw-sh wants to merge 5 commits intocloudflare:mainfrom
Conversation
Allow sending RST_STREAM with a custom reason code instead of hardcoded INTERNAL_ERROR. This enables RFC 7540 §9.1.2 compliant 421 responses where HTTP_1_1_REQUIRED can signal clients to retry over HTTP/1.1. Fixes cloudflare#787
Add shutdown_with_reason for H2 streams
Add a PreTlsProcess trait and set_pre_tls_callback() method that allows applications to process raw bytes before the TLS handshake occurs. This is useful for protocols like HAProxy's PROXY protocol, which sends client address information before TLS. The callback reads and consumes protocol headers, then updates the socket digest with the real client address.
PreTlsProcess implementations need to put data back onto the stream when it doesn't match the expected protocol signature. This lets the PROXY protocol handler rewind non-PROXY data so TLS proceeds normally.
78334c9 to
9aa73fb
Compare
drcaramelsyrup
left a comment
There was a problem hiding this comment.
This looks like a pretty straightforward optional callback hook to me, will report back if internal review reports any other suggested changes.
pingora-core/src/listeners/mod.rs
Outdated
| self.l4.set_buffer(); | ||
|
|
||
| // Process pre-TLS data if a callback is configured (e.g., PROXY protocol) | ||
| if let Some(ref callback) = self.pre_tls_callback { |
There was a problem hiding this comment.
@jaw-sh The name of the callback suggests that we might only want to do this if we will actually perform the tls_handshake a few lines below.
If the callback is only invoked when we actually intend on performing a TLS handshake, would that still work with your use case?
There was a problem hiding this comment.
Yes, for my purposes. In insecure HTTP, it is possible to intercept the origin's remote IP through standard header parsing such as Cloudflare's cf-connecting-ip.
There was a problem hiding this comment.
In that case I think it makes sense to move this callback inside the if let Some(tls) = self.tls block. "Pre-TLS" to the caller might mean that this is only attempted if we are going to invoke the TLS handshake afterwards.
This adds a
PreTlsProcesstrait and callback mechanism that allows applications to read and process bytes from the raw TCP stream before the TLS handshake begins. Provides a way to self-implement #132 .This also exposes HTTP2 session terminators. Closes #775.
The HAProxy PROXY protocol spec requires that the header be parsed from the raw TCP stream before any application protocol processing, including TLS. Currently there's no way to do this in Pingora - by the time
ServerApp::process_newruns, TLS has already completed.https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt
This is how nginx and haproxy handle it: TCP accept → parse PROXY header → TLS handshake → HTTP.
https://github.com/nginx/nginx/blob/master/src/http/ngx_http_request.c#L307-L362
Changes
PreTlsProcesstrait in pingora-core/src/listeners/mod.rsUninitializedStream::handshake()before TLSStream::rewind()public so implementations can put back bytes that aren't PROXY protocolUsage
This code is being used in a beta server deployment with PROXY Protocol v1 and v2 support to handle incoming traffic from other web servers and Tor with success.