After the file is uploaded to s3, the content is wrong, is it an encoding problem?

[shaojielinux]

After the file is uploaded to s3, the content is wrong, is it an encoding problem?
input {
kafka {
bootstrap_servers => "10.88.14.172:9092,10.88.6.9:9092,10.88.10.166:9092"
#topics => ["k8s"]
#group_id => "k8s1"
topics => ["fluent-log"]
#topics_pattern => ".*"
group_id => "k8s2"

#key_deserializer_class => "org.apache.kafka.common.serialization.ByteArrayDeserializer"
#value_deserializer_class => "org.apache.kafka.common.serialization.ByteArrayDeserializer"
consumer_threads => 4
decorate_events => true
auto_offset_reset => "latest"
codec => "json"
}
}
output {
s3{
access_key_id => ""
secret_access_key => ""
region => ""
bucket => ""
prefix => "%{[container_name]}/%{+YYYY}/%{+MM}/%{+dd}"
temporary_directory => "/tmp/logstash_s3"
encoding => "gzip"
codec => "plain"
rotation_strategy => "time"
time_file => 5
}
}

This is my configuration

s3 file content
2021-12-16T03:59:36.096Z %{host} %{message}2021-12-16T03:59:36.095Z %{host} %{message}2021-12-16T03:59:36.095Z %{host} %{message}2021-12-16T03:59:36.096Z %{host} %{message}2021-12-16T03:59:36.096Z %{host} %{message}2021-12-16T03:59:36.096Z %{host} %{message}2021-12-16T03:59:36.096Z %{host} %{message}2021-12-16T03:59:37.096Z %{host} %{message}2021-12-16T03:59:37.097Z %{host} %{message}2021-12-16T03:59:37.096Z %{host} %{message}2021-12-16T03:59:37.096Z %{host} %{message}2021-12-16T03:59:37.096Z %{host} %{message}2021-12-16T03:59:37.097Z %{host} %{message}2021-12-16T03:59:37.096Z %{host} %{message}2021-12-16T03:59:37.096Z %{host} %{message}2021-12-16T03:59:37.096Z %{host} %{message}2021-12-16T03:59:37.097Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:38.095Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:38.095Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:38.096Z %{host} %{message}2021-12-16T03:59:39.096Z %{host} %{message}2021-12-16T03:59:39.096Z %{host} %{message}2021-12-16T03:59:39.096Z %{host} %{message}2021-12-16T03:59:39.096Z %{host} %{message}2021-12-16T03:59:39.095Z %{host} %{message}2021-12-16T03:59:39.096Z %{host} %{message}2021-12-16T03:59:39.096Z %{host} %{message}2021-12-16T03:59:39.095Z %{host} %{message}2021-12-16T03:59:39.096Z %{host} %{message}2021-12-16T03:59:39.097Z %{host} %{message}2021-12-16T03:59:40.096Z %{host} %{message}2021-12-16T03:59:40.096Z %{host} %{message}2021-12-16T03:59:40.096Z %{host} %{message}2021-12-16T03:59:40.096Z %{host} %{message}2021-12-16T03:59:40.096Z %{host} %{message}

[roimor]

I am having the same issue. @shaojielinux did you find a workaround ?

[yaauie]

You appear to be using codec => plain in your output plugin. An output codec is responsible for converting a sequence of structured events into a sequence of bytes, and the Plain Codec is probably not what you are looking for.

If you want the events to be appended as-structured to a newline-delimited JSON stream, you likely should be using codec => json_lines.