Does BootDelegationInstrumentation cause NoClassDefFoundError?

[huange7]

I have made some modifications to the code of the agent. During the runtime, occasionally, the following issues occur.

2023-07-19 11:29:29.914 ERROR version_IS_UNDEFINED [DiscoveryClient-CacheRefreshExecutor-12]c.n.d.DiscoveryClient - DiscoveryClient_CUSTEMER_CENTER_FRONT/x.x.x.x:xxx - was unable to refresh its cache! status = io/opentelemetry/javaagent/shaded/io/opentelemetry/api/trace/DefaultTracerProvider948
java.lang.NoClassDefFoundError: io/opentelemetry/javaagent/shaded/io/opentelemetry/api/trace/DefaultTracerProvider
        at io.opentelemetry.javaagent.shaded.io.opentelemetry.api.trace.TracerProvider.noop(TracerProvider.java:24)
	at io.opentelemetry.sdk.trace.SdkTracer.spanBuilder(SdkTracer.java:34)
	at io.opentelemetry.javaagent.shaded.instrumentation.api.instrumenter.Instrumenter.spanBuilder(Instrumenter.java:423)
	at io.opentelemetry.javaagent.shaded.instrumentation.api.instrumenter.Instrumenter.doStart(Instrumenter.java:273)
	at io.opentelemetry.javaagent.shaded.instrumentation.api.instrumenter.Instrumenter.start(Instrumenter.java:150)
	at io.opentelemetry.javaagent.shaded.instrumentation.api.instrumenter.ClientInstrumenter.start(ClientInstrumenter.java:26)
	at com.sun.jersey.api.client.Client.handle(Client.java:646)
	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
	at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
	at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.getApplicationsInternal(AbstractJerseyEurekaHttpClient.java:194)
	at com.netflix.discovery.shared.transport.jersey.AbstractJerseyEurekaHttpClient.getDelta(AbstractJerseyEurekaHttpClient.java:170)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$7.execute(EurekaHttpClientDecorator.java:152)
	at com.netflix.discovery.shared.transport.decorator.MetricsCollectingEurekaHttpClient.execute(MetricsCollectingEurekaHttpClient.java:73)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.getDelta(EurekaHttpClientDecorator.java:149)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$7.execute(EurekaHttpClientDecorator.java:152)
	at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.executeOnNewServer(RedirectingEurekaHttpClient.java:118)
	at com.netflix.discovery.shared.transport.decorator.RedirectingEurekaHttpClient.execute(RedirectingEurekaHttpClient.java:79)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.getDelta(EurekaHttpClientDecorator.java:149)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$7.execute(EurekaHttpClientDecorator.java:152)
	at com.netflix.discovery.shared.transport.decorator.RetryableEurekaHttpClient.execute(RetryableEurekaHttpClient.java:119)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.getDelta(EurekaHttpClientDecorator.java:149)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator$7.execute(EurekaHttpClientDecorator.java:152)
	at com.netflix.discovery.shared.transport.decorator.SessionedEurekaHttpClient.execute(SessionedEurekaHttpClient.java:77)
	at com.netflix.discovery.shared.transport.decorator.EurekaHttpClientDecorator.getDelta(EurekaHttpClientDecorator.java:149)
	at com.netflix.discovery.DiscoveryClient.getAndUpdateDelta(DiscoveryClient.java:1061)
	at com.netflix.discovery.DiscoveryClient.fetchRegistry(DiscoveryClient.java:943)
	at com.netflix.discovery.DiscoveryClient.refreshRegistry(DiscoveryClient.java:1465)
	at com.netflix.discovery.DiscoveryClient$CacheRefreshThread.run(DiscoveryClient.java:1432)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

I have noticed that the BootDelegationInstrumentation class modifies the loadClass method to delegate the loading of certain classes to the bootstrap class loader. It uses CallDepth to prevent StackOverflow. However, could this potentially result in certain classes not being loaded by the bootstrap class loader and instead being handed over to the AppClassLoader or other class loaders?

I am confused about NoClassDefFoundError, Any answers are appreciated.

[laurit]

It would be helpful if you actually included the part of stack trace that has NoClassDefFoundError with the class name that was not found.

[huange7]

However, how can a Java program not load DefaultTracerProvider until it is closed?

Please ignore my question. I have discovered that this can indeed happen.

[huange7]

Hi, @laurit I have found that this situation can also occur in the release versions of OpenTelemetry. Do we need to fix this issue?

[laurit]

@huange7 if this is caused by removing/overwriting the agent jar then the best fix would be to stop doing that.

[xiepuhuan]

I have encountered this problem now and am looking for a solution.

[huange7]

this is caused by removing/overwriting the agent jar

It could be this reason.