GELF codec needs to support chunking

[neuronull]

I'm noticing that GELF does have a chunking part of its protocol too, for multi-part messages when using UDP:

Prepend the following structure to your GELF message to make it chunked:

Chunked GELF magic bytes - 2 bytes: 0x1e 0x0f
Message ID - 8 bytes: Must be the same for every chunk of this message. Identifying the whole message and is used to reassemble the chunks later. Generate from millisecond timestamp + hostname for example.
Sequence number - 1 byte: The sequence number of this chunk. Starting at 0 and always less than the sequence count.
Sequence count - 1 byte: Total number of chunks this message has.
All chunks MUST arrive within 5 seconds or the server will discard all already arrived and still arriving chunks. A message MUST NOT consist of more than 128 chunks.

I think we can split this off into a follow up issue though and just start with basic decoding support of incoming messages.

Originally posted by @jszwedko in #13288 (comment)

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[neuronull]

This should be attempted after #4868 is closed.

[paluchnuggets]

Any update on this matter?

[neuronull]

Any update on this matter?

👋 I've added it to our backlog. It currently doesn't have a high priority essentially due to not knowing how important it is to the community.

Please vote on this issue by adding a +1 reaction to the original comment on this issue to add your vote to prioritization. Thanks!

[angelrib]

@neuronull

I think we can split this off into a follow up issue though and just start with basic decoding support of incoming messages.

Does vector support chunking for output messages now? Or what you mean is start with the reassembling first?

Thank you

[neuronull]

Hi @angelrib , that comment actually came from @jszwedko (#13288 (comment))

So this (#13292) is the referred to follow-up issue to track chunking support.

I believe what Jesse meant is that we can probably just start with implementing chunking on the decoder side, and add the encoding side in a separate PR.

[angelrib]

Reviewing the code it looked like that, I just wanted to confirm it. Thank you very much!

[gernoteger]

Chunking imho is mandatory for UDP in order to be usable, since log messages would be limited to 8192 bytes otherwise, and fail depending on content.
Currently GELF is implemented as an encoding for various transports. Chunked GELF UDP has aspects of a transport protocol, like dependencies on packets and a defined timeout for a sequence. Maybe this would need dedicated source and sink components.

[jorgehermo9]

Opened #20769 for addressing the decoding part. I'm currently working on it and will have it implemented soon. Just some details left