snuba erroring out with SASL connection to Kafka not being supported only PLAIN?

[Alena86]

Self-Hosted Version

24.9.0

CPU Architecture

x86_64

Docker Version

26.0.2 build 3c863ff

Docker Compose Version

v2.26.1

Steps to Reproduce

in the docker compose file:

1. remove the kafka service (as we are using out own kafka setup)
2. under the x-snuba-defaults change the kafka settings from: DEFAULT_BROKERS: "kafka:9092" to have KAFKA settings:

 DEFAULT_BROKERS: "kafka.ourdomain.com:9094"
    KAFKA_SECURITY_PROTOCOL: "SASL_PLAINTEXT"
    KAFKA_SASL_MECHANISM: "SCRAM-SHA-512"
    KAFKA_SASL_USERNAME: "<username>"
    KAFKA_SASL_PASSWORD: "<password>"

3. Comment out wherever the service depends_on kafka service.

Tried even setting DEFAULT_BROKERS to be SASL_PLAINTEXT://kafka.ourdomain.com:9094 as we use 9094 port instead of 9092... even tried for the sake of it setting 9092 but the same error which would be the SASL not being supported in snuba container...

Expected Result

Expected result was to connect Sentry setup to our instance of Kafka and be able to open Sentry and login.

Actual Result

installation goes through, but once we run docker compose up, we are getting bunch of errors from snuba about not being able to connect to Kafka:

snuba-generic-metrics-counters-consumer-1       | note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
snuba-generic-metrics-counters-consumer-1       | Traceback (most recent call last):
snuba-generic-metrics-counters-consumer-1       |   File "/usr/local/bin/snuba", line 33, in <module>
snuba-generic-metrics-counters-consumer-1       |     sys.exit(load_entry_point('snuba', 'console_scripts', 'snuba')())
snuba-generic-metrics-counters-consumer-1       |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1       |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1130, in __call__
snuba-generic-metrics-counters-consumer-1       |     return self.main(*args, **kwargs)
snuba-generic-metrics-counters-consumer-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1       |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1055, in main
snuba-generic-metrics-counters-consumer-1       |     rv = self.invoke(ctx)
snuba-generic-metrics-counters-consumer-1       |          ^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1       |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1657, in invoke
snuba-generic-metrics-counters-consumer-1       |     return _process_result(sub_ctx.command.invoke(sub_ctx))
snuba-generic-metrics-counters-consumer-1       |                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1       |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 1404, in invoke
snuba-generic-metrics-counters-consumer-1       |     return ctx.invoke(self.callback, **ctx.params)
snuba-generic-metrics-counters-consumer-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1       |   File "/usr/local/lib/python3.11/site-packages/click/core.py", line 760, in invoke
snuba-generic-metrics-counters-consumer-1       |     return __callback(*args, **kwargs)
snuba-generic-metrics-counters-consumer-1       |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1       |   File "/usr/src/snuba/snuba/cli/rust_consumer.py", line 242, in rust_consumer
snuba-generic-metrics-counters-consumer-1       |     exitcode = rust_snuba.consumer(  # type: ignore
snuba-generic-metrics-counters-consumer-1       |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
snuba-generic-metrics-counters-consumer-1       | pyo3_runtime.PanicException: called `Result::unwrap()` on an `Err` value: KafkaError (Client creation error: No provider for SASL mechanism SCRAM-SHA-512: recompile librdkafka with libsasl2 or openssl support. Current build options: PLAIN)
snuba-outcomes-consumer-1                       |    0: rust_begin_unwind
snuba-outcomes-consumer-1                       |              at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:597:5
snuba-outcomes-consumer-1                       |    1: core::panicking::panic_fmt
snuba-outcomes-consumer-1                       |              at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/panicking.rs:72:14
snuba-outcomes-consumer-1                       |    2: core::result::unwrap_failed
snuba-outcomes-consumer-1                       |              at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1652:5
snuba-outcomes-consumer-1                       |    3: core::result::Result<T,E>::unwrap
snuba-outcomes-consumer-1                       |              at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/core/src/result.rs:1077:23
snuba-outcomes-consumer-1                       |    4: rust_arroyo::processing::StreamProcessor<rust_arroyo::backends::kafka::types::KafkaPayload>::with_kafka
snuba-outcomes-consumer-1                       |              at /root/.cargo/git/checkouts/arroyo-2fe52eb044a47b23/b15d546/rust-arroyo/src/processing/mod.rs:212:33
snuba-outcomes-consumer-1                       |    5: rust_snuba::consumer::consumer_impl
snuba-outcomes-consumer-1                       |              at ./rust_snuba/src/consumer.rs:257:21
snuba-outcomes-consumer-1                       |    6: rust_snuba::consumer::consumer::{{closure}}
snuba-outcomes-consumer-1                       |              at ./rust_snuba/src/consumer.rs:48:9
snuba-outcomes-consumer-1                       |    7: pyo3::marker::Python::allow_threads
snuba-outcomes-consumer-1                       |              at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/marker.rs:473:9
snuba-outcomes-consumer-1                       |    8: rust_snuba::consumer::consumer
snuba-outcomes-consumer-1                       |              at ./rust_snuba/src/consumer.rs:47:5
snuba-outcomes-consumer-1                       |    9: rust_snuba::consumer::_::__pyfunction_consumer
snuba-outcomes-consumer-1                       |              at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-consumer-1                       |   10: pyo3::impl_::trampoline::fastcall_with_keywords::{{closure}}
snuba-outcomes-consumer-1                       |              at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:41:35
snuba-outcomes-consumer-1                       |   11: pyo3::impl_::trampoline::trampoline_inner::{{closure}}
snuba-outcomes-consumer-1                       |              at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:54
snuba-outcomes-consumer-1                       |   12: std::panicking::try::do_call
snuba-outcomes-consumer-1                       |              at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:504:40
snuba-outcomes-consumer-1                       |   13: std::panicking::try
snuba-outcomes-consumer-1                       |              at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panicking.rs:468:19
snuba-outcomes-consumer-1                       |   14: std::panic::catch_unwind
snuba-outcomes-consumer-1                       |              at /rustc/a28077b28a02b92985b3a3faecf92813155f1ea1/library/std/src/panic.rs:142:14
snuba-outcomes-consumer-1                       |   15: pyo3::impl_::trampoline::trampoline_inner
snuba-outcomes-consumer-1                       |              at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:204:9
snuba-outcomes-consumer-1                       |   16: pyo3::impl_::trampoline::fastcall_with_keywords
snuba-outcomes-consumer-1                       |              at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pyo3-0.18.3/src/impl_/trampoline.rs:52:1
snuba-outcomes-consumer-1                       |   17: rust_snuba::consumer::_::<impl rust_snuba::consumer::consumer::MakeDef>::DEF::trampoline
snuba-outcomes-consumer-1                       |              at ./rust_snuba/src/consumer.rs:27:1
snuba-outcomes-consumer-1                       |   18: <unknown>
snuba-outcomes-consumer-1                       |   19: PyObject_Vectorcall
snuba-outcomes-consumer-1                       |   20: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1                       |   21: <unknown>
snuba-outcomes-consumer-1                       |   22: <unknown>
snuba-outcomes-consumer-1                       |   23: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1                       |   24: <unknown>
snuba-outcomes-consumer-1                       |   25: <unknown>
snuba-outcomes-consumer-1                       |   26: <unknown>
snuba-outcomes-consumer-1                       |   27: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1                       |   28: <unknown>
snuba-outcomes-consumer-1                       |   29: <unknown>
snuba-outcomes-consumer-1                       |   30: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1                       |   31: <unknown>
snuba-outcomes-consumer-1                       |   32: _PyObject_FastCallDictTstate
snuba-outcomes-consumer-1                       |   33: _PyObject_Call_Prepend
snuba-outcomes-consumer-1                       |   34: <unknown>
snuba-outcomes-consumer-1                       |   35: _PyObject_MakeTpCall
snuba-outcomes-consumer-1                       |   36: _PyEval_EvalFrameDefault
snuba-outcomes-consumer-1                       |   37: <unknown>
snuba-outcomes-consumer-1                       |   38: PyEval_EvalCode
snuba-outcomes-consumer-1                       |   39: <unknown>
snuba-outcomes-consumer-1                       |   40: <unknown>
snuba-outcomes-consumer-1                       |   41: <unknown>
snuba-outcomes-consumer-1                       |   42: _PyRun_SimpleFileObject
snuba-outcomes-consumer-1                       |   43: _PyRun_AnyFileObject
snuba-outcomes-consumer-1                       |   44: <unknown>
snuba-outcomes-consumer-1                       |   45: Py_BytesMain
snuba-outcomes-consumer-1                       |   46: <unknown>
snuba-outcomes-consumer-1                       |   47: __libc_start_main
snuba-outcomes-consumer-1                       |   48: _start

it comes from almost all the snuba components the same error.

Is snuba image really not supporting Kafka SASL?
And I thought snuba is supposed to be connecting only to clickhouse? Why does it even need kafka configuration?

Thanks for the help!

Event ID

No response

[aldy505]

It's not supported, see here getsentry/snuba#6358 (comment)

I'm not an employee, so I suggest creating another issue at the snuba repo.

[Alena86]

Got ya..
Is this the snuba repo you are reffering to?

[aldy505]

Yes that's the one!

[onkar]

Duplicate of getsentry/snuba#6441. Let's keep only one issue open.