Request to add environment variable for Span size limitation

[leon4652]

Is your feature request related to a problem? Please describe.

Hello, I would like to suggest an idea inspired by a problem that occurred during the process of Agent and Span collection. It would be helpful if there were an environment variable or feature to limit the size of Span objects.

During the additional Span and Attributes collection process, a memory leak issue occurred. This was caused by the Span collected by the Agent, and the Old Gen region significantly increased throughout the collection period.

I suspect that this issue happened due to 'large-sized Span - where the content of Attributes grew longer, causing premature promotion from Eden to Old Gen.' Upon examining the collected data, I found that a number of abnormal Spans exceeding a certain size were collected. (Of course, this was caused by my individual Extension and collection logic.) I suspect that this is what triggered the memory leak.

In fact, I didn’t anticipate the Span size would grow so large until I analyzed it using tools, which led to the server overload.

Heap Used Graph(Metric), Max Heap size 8gb

Major GC occurred

Describe the solution you'd like

Could you please consider adding an option to set a maximum size for individual Span instances through environment variables? For instance, if a Span exceeds the specified size in KB, it could be automatically discarded or the memory could be released immediately. This feature could also be configured based on GC settings (e.g., XX:PretenureSizeThreshold).

Describe alternatives you've considered

As described above.

Additional context

I find this Agent very useful. Thank you for reading.

[laurit]

While it is not possible to limit span size it is possible to limit the number of attributes and their size, see https://opentelemetry.io/docs/languages/java/configuration/#properties-general Limiting based on span size doesn't seem too useful to me because the span size becomes know only after it has been serialized by the exporter, so a lot of work would need to be done before such limit could be applied.

During the additional Span and Attributes collection process, a memory leak issue occurred.

Usually a memory leak means that memory is accumulated somewhere and never freed, https://en.wikipedia.org/wiki/Memory_leak. I assume here you must be meaning something else when you say memory leak. If you really observed a memory leak then limiting the span size would be unlikely to fix it.

I suspect that this issue happened

Usually issues that present strong evidence rather than a gut feeling are more likely to get resolved.

Looking at the screenshot from memory analyzer the first thing I'd note is that we have done some work to lower the allocation rate in exporters. See https://opentelemetry.io/docs/languages/java/configuration/#properties-exporters setting the experimental memory mode option to reusable_data could help. Secondly I'd interpret the screenshot so that you are exporting 120537 span with total size of 78mb, the average size of a span is 650 bytes. Is my understand correct? To me the average size of a span doesn't seem large. On the contrary the total number of spans is large. The default exported batch size 512, have you increased it?

Finally I'd like to note that the exporters are in https://github.com/open-telemetry/opentelemetry-java if you so wish we can move this issue there.