Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

promtail - Geoip always sets its default labels #15418

Open
replay111 opened this issue Dec 14, 2024 · 0 comments
Open

promtail - Geoip always sets its default labels #15418

replay111 opened this issue Dec 14, 2024 · 0 comments
Labels
component/promtail type/feature Something new we should do

Comments

@replay111
Copy link

Describe the bug
I have lines from firewall, where I have two fields with IP addreses: src_ip, dest_ip.
I have prepared configuration that calls twice geoip and sets custom labels, but result is that geoip always sets its own labels, and always for the last instance in pipeline.
So I did test by setting in pipeline only one call of geoip and adding my custom labels - still result is the same - all geoip labels are added by default, my label is not even visible

To Reproduce
config with two geoip:

server:
  http_listen_port: 9080
  http_listen_address: 0.0.0.0
  grpc_listen_port: 9095
  # log_level: info
  log_level: debug

positions:
  filename: /data/positions.yaml


clients:
  - url: https://loki.local/loki/api/v1/push
    tls_config:
      insecure_skip_verify: true

scrape_configs:

  - job_name: mikrotik
    syslog:
      syslog_format: "rfc3164"
      listen_address: 0.0.0.0:6044
      idle_timeout: 20s
      label_structured_data: true
      listen_protocol: udp
      labels:
        job: "syslog_mikrotik"
    pipeline_stages:
      - regex:
          expression: '(?P<src_ip>\d{1,3}(?:\.\d{1,3}){3}):(?P<src_port>\d+)->(?P<dest_ip>\d{1,3}(?:\.\d{1,3}){3}):(?P<dest_port>\d+)'
      - labels:
          src_ip:
          src_port:
          dest_ip:
          dest_port:
      - geoip:
          db: "/assets/geo_db/GeoLite2-City.mmdb"
          db_type: "city"
          source: src_ip
      - labels:
          src_geoip_city_name: '{{ .geoip_city_name }}' 
          src_geoip_country_name: '{{ .geoip_country_name }}'
          src_geoip_continent_name: '{{ .geoip_continent_name }}'
          src_geoip_continent_code: '{{ .geoip_continent_code }}'
          src_geoip_location_latitude: '{{ .geoip_location_latitude }}'
          src_geoip_location_longitude: '{{ .geoip_location_longitude }}'
          src_geoip_postal_code: '{{ .geoip_postal_code }}'
          src_geoip_timezone: '{{ .geoip_timezone }}'
          src_geoip_subdivision_name: '{{ .geoip_subdivision_name }}'
          src_geoip_subdivision_code: '{{ .geoip_subdivision_code }}'
      - geoip:
          db: "/assets/geo_db/GeoLite2-City.mmdb"
          db_type: "city"
          source: dest_ip
      - labels:
          dest_geoip_city_name: '{{ .geoip_city_name }}' 
          dest_geoip_country_name: '{{ .geoip_country_name }}'
          dest_geoip_continent_name: '{{ .geoip_continent_name }}'
          dest_geoip_continent_code: '{{ .geoip_continent_code }}'
          dest_geoip_location_latitude: '{{ .geoip_location_latitude }}'
          dest_geoip_location_longitude: '{{ .geoip_location_longitude }}'
          dest_geoip_postal_code: '{{ .geoip_postal_code }}'
          dest_geoip_timezone: '{{ .geoip_timezone }}'
          dest_geoip_subdivision_name: '{{ .geoip_subdivision_name }}'
          dest_geoip_subdivision_code: '{{ .geoip_subdivision_code }}'

    relabel_configs:
      - source_labels: ['__syslog_message_severity']
        target_label: 'severity'
      - source_labels: ['__syslog_message_hostname']
        target_label: 'host'
      - source_labels: ['__syslog_message_app_name']
        target_label: 'app'  
        

in this setup only dest_ip is tested and included with default labels.
When I remove part:

      - geoip:
          db: "/assets/geo_db/GeoLite2-City.mmdb"
          db_type: "city"
          source: dest_ip
      - labels:
          dest_geoip_city_name: '{{ .geoip_city_name }}' 
          dest_geoip_country_name: '{{ .geoip_country_name }}'
          dest_geoip_continent_name: '{{ .geoip_continent_name }}'
          dest_geoip_continent_code: '{{ .geoip_continent_code }}'
          dest_geoip_location_latitude: '{{ .geoip_location_latitude }}'
          dest_geoip_location_longitude: '{{ .geoip_location_longitude }}'
          dest_geoip_postal_code: '{{ .geoip_postal_code }}'
          dest_geoip_timezone: '{{ .geoip_timezone }}'
          dest_geoip_subdivision_name: '{{ .geoip_subdivision_name }}'
          dest_geoip_subdivision_code: '{{ .geoip_subdivision_code }}'

in loki I see geoip data with default (not mine) labels and for src_ip....

Expected behavior
get set of two data of geolocation prefixed by src_ and dest_.

Environment:

  • docker
  • manual
@JStickler JStickler added type/feature Something new we should do component/promtail labels Dec 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component/promtail type/feature Something new we should do
Projects
None yet
Development

No branches or pull requests

2 participants