Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: change the default value of initial_max_send_streams to 100 #731

Open
magurotuna opened this issue Jan 7, 2024 · 7 comments · Fixed by Cysharp/YetAnotherHttpHandler#71

Comments

@magurotuna
Copy link
Contributor

magurotuna commented Jan 7, 2024

Issue

Currently, the number of streams opened by the client before it receives the initial SETTINGS frame from the server is set to usize::MAX by default.

initial_max_send_streams: usize::MAX,

This is valid according HTTP/2 spec (in fact the spec doesn't mention any explicit limit here). However, opening unlimited number of streams can lead to majority of them being rejected by the server with REFUSED_STREAM, if the server's SETTINGS_MAX_CONCURRENT_STREAMS is set to any lower value. This is simply not ideal and waste of resources.

Proposal

To better handle this issue, I propose that we start with reasonably low number of streams by default, rather than the maximum possible number. The initial number is only used until the client receives the initial SETTINGS frame, so it should be much more preferable to start with a low number in order to reduce the risk of REFUSED_STREAM errors even if it might end up with lower utilization for a very short time.

What is the reasonable default number? The good number should be as low as what most servers would accept. In fact, the spec recommends that the value should be no smaller than 100. To back up this recommendation, I've done a quick research on the values of MAX_CONCURRENT_STREAMS used by several popular servers. The result is as follows.

Server MAX_CONCURRENT_STREAMS
google.com 100
youtube.com 100
x.com 100
facebook.com 100
www.amazon.com 100
www.apple.com 100
netflix.com 100
microsoft.com 100
cloudflare.com 100
fastly.com 100
hyper.rs 100
openai.com 128
www.rust-lang.org 128
crates.io 128
vercel.com 250
tokio.rs 256
deno.com (unspecified)

So if the client starts with 100 streams at maximum, all of these servers will accept them and the client will not receive any REFUSED_STREAM errors. Therefore we can conclude that 100 is the reasonable default number of streams to start with.

Take a quick look at other HTTP/2 implementations

To further support the proposal, let's have a look at how other HTTP/2 client implementations are doing. In short, all of these three implementations adopt starting with the reasonably small number of streams.

nghttp2

nghttp2's default initial maximum number of concurrent streams is 100.
https://github.com/nghttp2/nghttp2/blob/837f0c67c7139aedfb0c31d252634c823bfa75ba/lib/includes/nghttp2/nghttp2.h#L2537-L2554

Go x/net/http2

Go also uses 100 as the the initial number of maximum concurrent strams.
https://cs.opensource.google/go/x/net/+/689bbc7005f6bbf9fac1a8333bf03436fa4b4b2a:http2/transport.go;l=56

One notable difference from nghttp2 is that Go will set 1,000 as the MAX_CONCURRENT_STREAMS if the server doesn't provide one, while nghttp2 sets inifinity in this case.

Python httpx

httpx starts with just 1.
https://github.com/encode/httpcore/blob/2fcd062df71555cc7de55774c6dc137551eb8692/httpcore/_async/http2.py#L117-L119

@seanmonstar
Copy link
Member

This is a great write-up, thank you! I agree it makes sense to adjust the default. I only want to pause to make sure it's the right place to adjust the default.

It's worth calling out that for workloads where the server will NOT place a limit, changing this default here could be a negative. This would be places where both peers are trusted. My mind goes to things like linkerd2 proxies. (cc @hawkw @olix0r) The proper solution could actually be that those use cases should explicitly set a high initial value, instead of depending on the default, which perhaps should aim to protect the most common cases.

But I also vaguely feel like so far we've put those defaults in hyper or reqwest, and left h2 to simply implement the protocol, and otherwise not pick defaults.

@magurotuna
Copy link
Contributor Author

magurotuna commented Jan 9, 2024

It makes sense that changing the default value could have a negative impact in a rare situation.
Actually, I opened an issue to here instead of to hyper (or reqwest) because I found that the client builder in h2 provides a way to set the initial MAX_CONCURRENT_STREAMS value while hyper doesn't. Plus, it looks like hyper doesn't set any value for initial MAX_CONCURRENT_STREAMS when constructing h2 client, thus ending up using the default value specified in h2, i.e. usize::MAX.

So probably what we would like to do is:

I also feel that some changes still need to be made to h2, namely:

  • improves the doc of h2::client::Builder::initial_max_send_streams to clearify that the default value is usize::MAX
  • sets MAX_CONCURRENT_STREAMS to usize::MAX or whatever reasonable value (for example Go adopts 1,000) if the server's initial SETTINGS frame doesn't include MAX_CONCURRENT_STREAMS, assuming that the absence of MAX_CONCURRENT_STREAMS implies the server is willing to accept infinite number of concurrent streams

Do these sound reasonable to you? If they do, I'd be happy to work on these :)

@dswij
Copy link
Member

dswij commented Jan 9, 2024

But I also vaguely feel like so far we've put those defaults in hyper or reqwest, and left h2 to simply implement the protocol, and otherwise not pick defaults.

+1. h2 definitely should concern the protocol-level implementation only. The defaults should be in hyper.

Having arbitrary defaults everywhere is also a nightmare to handle, so it's good that we're avoiding that.

to have hyper call h2::client::Builder::initial_max_send_streams when constructing h2 client with the user-configured value, if any.

This sounds like it should be exposed in hyper

@magurotuna
Copy link
Contributor Author

Thanks for your opinion @dswij.

This sounds like it should be exposed in hyper

What do you mean by this? If what you mean is that the users of hyper should be able to configure the initial value of max send streams, then I think it's covered by the first bullet:

@dswij
Copy link
Member

dswij commented Jan 9, 2024

What do you mean by this? If what you mean is that the users of hyper should be able to configure the initial value of max send streams, then I think it's covered by the first bullet

Yes :)

I'm completely on board with the first point. Just saying that regardless of changing the default value, this option should be configurable in hyper. So I see this as another improvement if we go down this route.

@magurotuna
Copy link
Contributor Author

So I separated #732 into a couple of PRs. Now the part of setting a reasonable default value is moved to hyper. Here are the separated PRs:

@seanmonstar
Copy link
Member

Thanks a lot for doing that, your contributions have been wonderful to work with ♥️

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants