Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

large memory usage increment from about 300M to 10G since v2.1.9 (more than 10 times of v2.1.8) #9012

Open
hsingli20 opened this issue Jun 26, 2024 · 1 comment

Comments

@hsingli20
Copy link

Bug Report

Describe the bug
Large memory usage increment from about 300M to 10G since v2.1.9 (more than 10 times of v2.1.8)
The memory of container will keep increasing until close to 10G. Memory increases in a sawtooth pattern and then releases.
Under the same traffic test and the same configurations for fluentbit, the version v2.1.9 seems introducing such new memory usage pattern, this will cause more memory usage reserved for fluentbit to avoid OOM kill issue.

To Reproduce

  • Rubular link if applicable: 100%

  • Steps to reproduce the problem:

  1. stress the logging traffic
  2. monitor the memory usage
    Expected behavior

Screenshots

Your Environment

  • Version used: v2.1.9
  • Configuration:
[INPUT]
    name                tail
    tag                 event.kafka.ingress
    alias               kafka.ingress
    buffer_chunk_size   1m
    buffer_max_size     1m
    read_from_head      true
    refresh_interval    5
    rotate_wait         10
    skip_empty_lines    off
    skip_long_lines     true
    key                 message
    db                  /var/log/kafka.ingress.db
    db.sync             normal
    db.locking          true
    db.journal_mode     off
    db.compare_filename true
    path                /var/log/app/*
    exclude_path        /var/log/app/xxx.log,/var/log/app/ingress/*.gz,/var/log/app/ingress/*.tgz
    mem_buf_limit       20MB
    parser              json
    ignore_older        11m

  • Environment name and version (e.g. Kubernetes? What version?): kubernetes
    MicrosoftTeams-image (1)
@edsiper
Copy link
Member

edsiper commented Jun 26, 2024

please provide the full configuration including filters and outputs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants