Shading Caffeine into instrumentation-api

[anuraaga]

I've been recently looking into how we can separate library instrumentation for DB, e.g. mongodb or lettuce. I found that our current sanitization is in javaagent-api

https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/javaagent-api/src/main/java/io/opentelemetry/javaagent/instrumentation/api/db

And it's not trivial to move to instrumentation-api because it depends on BoundedCache

https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/javaagent-api/src/main/java/io/opentelemetry/javaagent/instrumentation/api/BoundedCache.java

While technically it would work, it would not have any cache, which I guess isn't really acceptable for library instrumentation performance. Does it make sense to shade Caffeine, or similar utilities, into instrumentation-api which ends up in bootstrap classloader? Or any other ideas on how to implement this sort of logic to work well with both agent and library?

[iNikem]

Do we need "proper cache" at all? Or we can just use LinkedHashMap with properly overridden removeEldestEntry for simple LRU cache?

[anuraaga]

We'd need to benchmark the fact that we'd need locks for using JDK for these caches I believe. Maybe it doesn't matter too much, but I think our usage of caches does seem like it could be with fair amount of contention. But it's definitely a good idea to compare, want to confirm I didn't miss any other options too.

[whiskeysierra]

@ben-manes Can you share your benchmark results here?

[anuraaga]

Ah - didn't realize we had LinkedHashMap_lru. Maybe I don't need to benchmark since it looks pretty terrible :-)

https://github.com/ben-manes/caffeine/wiki/Benchmarks#read-100

[ben-manes]

https://github.com/ben-manes/caffeine/wiki/Benchmarks

If you just need a concurrent LinkedHashMap then you might copy that project into yours. It is tiny, was used broadly, and still widely embedded.

[anuraaga]

@ben-manes I think you're referring to https://github.com/ben-manes/concurrentlinkedhashmap? Thanks a lot for the reference, didn't know there was such a tiny but seemingly still pretty fast map. Definitely something to keep in mind if we need to make a tradeoff on size vs speed.

[ben-manes]

Yes. I wrote it in the Java 5 days, the Cassandra folks found it, and banged on it plenty. Groovy has a fork that added compute support and SqlServer drivers embed a copy for statement caching. It is bare bones as I was trying to figure out how a concurrent lru might work, whereas caffeine is feature rich but bloated. It seems like a promising fit for you.

[ben-manes]

I noticed you are using Guava for the current lru. At the default concurrency level, 4, it has similar performance to a synchronized LHM. The benchmarks are at 64 to be fair by showing it at a performance tuned setting.

The only benefit is concurrent loads, which a LHM doesn’t support. You can simulate it enough by lock striping and double checked locking. If the current cache is good enough then you don’t need any libraries and can use LHM instead.