Clickhouse issue after upgrade to 24.7.0, can be because of rust-consumer

[karatekaneen]

Self-Hosted Version

24.7.0

CPU Architecture

x86_64

Docker Version

26.1.3

Docker Compose Version

2.27.0

Steps to Reproduce

1. Run install.sh
2. Add some traffic
3. Go to UI only to see no issues or performance traces.

Expected Result

I expect issues and performance tracing to work.

I tried resetting Kafka (from this response) since it was an old instance that we upgraded to 24.7.0 and that fix has worked for similar issues in the past when we stopped receiving events. It did not work.

I also tried switching from rust-consumer to consumer in the Docker compose file and this did fix the issue. However, I guess this is more of a temporary fix and not feasible longterm.

Actual Result

Clickhouse logs

2024.08.22 08:59:54.107661 [ 1 ] {} <Warning> Application: Listen [::]:9005 failed: Poco::Exception. Code: 1000, e.code() = 0, DNS error: EAI: Address family for hostname not supported (version 23.8.11.29.altinitystable (altinity build)). If it is an IPv6 or IPv4 address and your host has disabled IPv6 or IPv4, then consider to specify not disabled IPv4 or IPv6 address to listen in <listen_host> element of configuration file. Example for disabled IPv6: <listen_host>0.0.0.0</listen_host> . Example for disabled IPv4: <listen_host>::</listen_host>
2024.08.22 09:02:07.412610 [ 47 ] {} <Error> ServerErrorHandler: Poco::Exception. Code: 1000, e.code() = 107, Net Exception: Socket is not connected, Stack trace (when copying this message, always include the lines below):

0. Poco::Net::SocketImpl::error(int, String const&) @ 0x0000000015b3dbf2 in /usr/bin/clickhouse
1. Poco::Net::SocketImpl::peerAddress() @ 0x0000000015b40376 in /usr/bin/clickhouse
2. DB::HTTPServerRequest::HTTPServerRequest(std::shared_ptr<DB::IHTTPContext>, DB::HTTPServerResponse&, Poco::Net::HTTPServerSession&) @ 0x0000000013154417 in /usr/bin/clickhouse
3. DB::HTTPServerConnection::run() @ 0x0000000013152ba4 in /usr/bin/clickhouse
4. Poco::Net::TCPServerConnection::start() @ 0x0000000015b42834 in /usr/bin/clickhouse
5. Poco::Net::TCPServerDispatcher::run() @ 0x0000000015b43a31 in /usr/bin/clickhouse
6. Poco::PooledThread::run() @ 0x0000000015c7a667 in /usr/bin/clickhouse
7. Poco::ThreadImpl::runnableEntry(void*) @ 0x0000000015c7893c in /usr/bin/clickhouse
8. ? @ 0x00007f43579da609 in ?
9. ? @ 0x00007f43578ff353 in ?
 (version 23.8.11.29.altinitystable (altinity build))

Relay error logs

2024-08-22T09:04:04.666097Z ERROR relay_server::services::project_upstream: error fetching project state 124c98746b404623b39d7c13e9d34e5b: deadline exceeded errors=0 pending=168 tags.did_error=false tags.was_pending=true tags.project_key="124c98746b404623b39d7c13e9d34e5b"

Event ID

No response

[hubertdeng123]

This is a duplicate of getsentry/snuba#5707, so I'm going to close this.