[Feature Request] Larger Object Pool

[hyperxpro]

Stormpot is going great for my Load Balancer but sometimes during peak usages, the object pool is hitting 1.2B objects. Stormpot uses int so the maximum object pool size is limited to 2B.

Do we have plans to support a larger object pool (greater than 2B) using long?

[chrisvest]

That's a lot of objects. There's no way to have objects returned sooner to reuse faster, or otherwise reduce the scale?

Besides switching to long being an API change, it's not difficult to do, but I still question if it's really a good idea to allow pools to get that big. Can you explain the use case a bit more?

[hyperxpro]

I implemented Stormpot in my Load Balancer. For every request (HTTP) hitting Load Balancer, few objects are created such as Maps, classes holding Upstream and Downstream connection information, etc.

During peak load times when Load Balancer is handling 10M+ requests, object allocation rate shoots up and so GC time. After the implementation of Stormpot, I saw a crazy 70% drop in object allocation rate during peak load times. And so, GC spent less time. However, one-day Stormpot made a high of 1.2B pool size when LB was handling 14.6M RPS. Since then, LB never got such traffic but I want to be future proof.

I agree that changing int to long would break API. But I want to know possible solutions. Should I fork and shade Stormpot?

[hyperxpro]

LB creates objects every request and once the request is forwarded to the backend, those objects are reusable for new requests. LB takes a few nanoseconds to forward requests to the backend. And allocating few objects for few nanoseconds and then destroying them is definitely not cheap.

Stormpot is a savior. :D

[chrisvest]

Still surprising the number of objects would get that high, but okay. The setTargetSize can be overloaded to both take int and long arguments. The getTargetSize cannot be overloaded so that would have to get a sibling called something like getTargetSizeAsLong.

I'm still waiting for approval to continue working on this project, so what you could do is make a PR with these changes, and then use that branch internally until I'm able to merge it.

[hyperxpro]

Makes sense. I will open PR soon.

[chrisvest]

Do you have leak detection enabled with such large pools?

I'm also thinking the more sensible approach would be to wrap multiple internal pools.

[hyperxpro]

Nope, I kept it disabled because I found leak detection was causing unlinear latency.

[chrisvest]

Figures. The algorithm used in 2.x & 3.x isn't scalable at all. Should be dramatically better for large pools in 4.x with #178, but at the size you're working with the extra objects will also have a non-trivial impact on the heap.