Graceful handling of empty :authority header

[htuch]

Today, when nghttp2 observes an empty :authority, it rejects the HEADERS frame and resets the stream. It seems this is problematic, since it seems legitimate to have an empty :authority, similar to an empty host, see https://tools.ietf.org/html/rfc7230#section-5.4.

Should we have a default authority configurable for HCM that is used when :authority is empty? My inclination is to add this.

When a default authority is not configured, should we have Envoy do something sensible (pick default virtual host, pick first virtual host, etc.) or continue to reject the stream? I don't have a strong feeling on this one.

For context, we've seen this situation arise in one of our integration test probers.

[mattklein123]

Today, when nghttp2 observes an empty :authority, it rejects the HEADERS frame and resets the stream

Did you debug where this is happening? I don't see any obvious check along these lines here: https://github.com/nghttp2/nghttp2/blob/master/lib/nghttp2_http.c which is where nghttp2 does its section 8 checks.

Should we have a default authority configurable for HCM that is used when :authority is empty? My inclination is to add this. When a default authority is not configured, should we have Envoy do something sensible (pick default virtual host, pick first virtual host, etc.) or continue to reject the stream? I don't have a strong feeling on this one.

Personally I wouldn't bother unless someone asks for it, but we should probably at least send a 400 or 404 or whatever we do when its not present in an HTTP/1 request. But either way first we would need to get nghttp2 to not reject it.

[htuch]

I think it's happening https://github.com/nghttp2/nghttp2/blob/d93842db3eb481fc6953ec70a81ed2f7863ffb45/lib/nghttp2_http.c#L90.

We might want this feature to ensure we provide compatible semantics between Envoy and non-Envoy proxies.

[alyssawilk]

@PiotrSikora as our spec expert :-)

We have a default host for HTTP/1.0 where it's explicitly allowed. I think we reject empty-but-present hosts for 1.1 and I have no idea if it's even allowed for 2.0 but I bet Piotr does!

Also if empty authority is spec-legal I suspect we'd need an nghttp2 fix_update before we can do anything useful in Envoy

[alyssawilk]

FWIW we use default host for all versions of HTTP in-google (because existing behavior is hard to change) and I'd frankly love a configurable option to continue flouting the spec to not cause customers major pain as they adopt Envoy. :-D
May be harder to sell nghttp2 folk if it's not allowed though

[mattklein123]

I think it's happening https://github.com/nghttp2/nghttp2/blob/d93842db3eb481fc6953ec70a81ed2f7863ffb45/lib/nghttp2_http.c#L90.

If it's hitting this, you aren't sending an empty header, the value must only be LWS, right?

May be harder to sell nghttp2 folk if it's not allowed though

If I read the code it is allowed, it just can't start with LWS. That's from a quick skim.

[htuch]

If it's hitting this, you aren't sending an empty header, the value must only be LWS, right?

The lws function validates that there is at least one non-whitespace character in the value. An empty string would fail this validation.

[mattklein123]

The lws function validates that there is at least one non-whitespace character in the value. An empty string would fail this validation.

Ah I see. Yes. Yeah I have no idea about the spec on this one.

[htuch]

@tatsuhiro-t would you be able to weigh in on how nghttp2 considers this corner of the H2 RFC?

[PiotrSikora]

I believe this is where it gets rejected by nghttp2, though I didn't verify it.

Per RFC7540, sec. 8.1.2.3, a missing :authority pseudo-header is allowed:

To ensure that the HTTP/1.1 request line can be reproduced
accurately, this pseudo-header field MUST be omitted when
translating from an HTTP/1.1 request that has a request target in
origin or asterisk form (see [RFC7230], Section 5.3).

...but the "origin-form" doesn't apply to proxies (so it's illegal in Envoy, at least in theory), and the "asterisk-form" applies only to OPTIONS * HTTP/1.1, as far as I know.

I don't believe that an empty :authority is allowed, though.

What's the "legitimate" request that has an empty :authority, @htuch? Can you share the whole request (and ideally info on the user-agent)?

[htuch]

@PiotrSikora it's custom prober code. The point is that we (Google) need to maintain strict compatibility between our Envoy and non-Envoy edge proxies, including corner case behavior like this. I don't see where an empty :authority is disallowed, so unless there is an RFC argument to disallow this (and even then), we would like some option to support the behavior.

[PiotrSikora]

Sorry, got confused by the "origin-form" for a second, per RFC7230, sec. 5.3.1, "origin-form" should still include the Host header:

A Host header field is also sent, as defined in Section 5.4.

though it could be empty, per sec. 5.4:

If the authority component is missing or
undefined for the target URI, then a client MUST send a Host header
field with an empty field-value.

So it seems that it's an empty Host header in HTTP/1.1 and omitted :authority header in HTTP/2.

[PiotrSikora]

@htuch per RFC7540, sec. 8.1.2.3:

The ":authority" pseudo-header field includes the authority
portion of the target URI ([RFC3986], Section 3.2). The authority
MUST NOT include the deprecated "userinfo" subcomponent for "http"
or "https" schemed URIs.

The "authority" is defined in RFC3986, sec. 3.2 as:

authority = [ userinfo "@" ] host [ ":" port ]

which with the HTTP/2 clause about deprecated "userinfo" becomes:

authority = host [ ":" port ]

"host" is further defined in sec. 3.2.2 as:

host = IP-literal / IPv4address / reg-name

and an empty value is declared as invalid for the "http" scheme:

If the URI scheme defines a default for host, then that default
applies when the host subcomponent is undefined or when the
registered name is empty (zero length). For example, the "file" URI
scheme is defined so that no authority, an empty host, and
"localhost" all mean the end-user's machine, whereas the "http"
scheme considers a missing authority or empty host invalid.

I'm not sure if there is more authoritative document for that.

[tatsuhiro-t]

I don't see any reason why empty :authority header field should be allowed. It seems that HTTP/2 specification only requires :authority header field if it is not empty. Does any browser send empty :authority header field? Or some intermediaries failed to produce proper HTTP/2 header field from HTTP/1.x request?
That said, RFC is not super precise about this topic. It might be better to ask HTTPWG about it.

[stale]

This issue has been automatically marked as stale because it has not had activity in the last 30 days. It will be closed in the next 7 days unless it is tagged "help wanted" or other activity occurs. Thank you for your contributions.