No traffic analyze on kafka interface

[atemix]

ntopng 6.1.240603 (Ubuntu 22.04.4 LTS)

Hi @lucaderi
Reconfigured nprobes in #8425.
https://www.ntop.org/guides/nprobe/kafka.html#exporting-flows-to-ntopng

#cat /etc/ntopng/ntopng.conf | grep kafka
--interface="kafka://127.0.0.1:9091"
#cat /etc/nprobe/nprobe-netflow.conf | grep kafka
--ntopng="kafka://127.0.0.1:9091"
#cat /etc/nprobe/nprobe-sflow.conf | grep kafka
--ntopng="kafka://127.0.0.1:9091"

No change, ntopng shows No traffic yet.

On ntopng restart for a couple of seconds appears traffic breakdown pie and exported flow count, but after again No traffic yet.
If all nprobes stopped and no new messages in kafka (0 bps traffic in ntopng), ntopng starts to show traffic and networks. When traffic resumes, breakdown and exported flows counter hangs and doesn't changes.

Nprobe creates 8 topics in kafka, but 4 of them remain empty. Is it normal?
counter, listening-ports, snmp-ifaces, template.

In ntopng log repeated messages:

04/Jun/2024 09:43:09 [KafkaCollectorInterface.cpp:41] ERROR: [thrd:app]: rdkafka#consumer-1: message [0]: topic does not exist (Broker: Unknown topic or partition)
04/Jun/2024 09:43:09 [KafkaCollectorInterface.cpp:41] ERROR: [thrd:app]: rdkafka#consumer-1: message [0]: topic does not exist (Broker: Unknown topic or partition)

[lucaderi]

Yes it is normal that some topics are not used. CAn you please check if nprobe in injecting flows into kafka and ntopng is unable to read them, or if the problem is that in your case nprobe does not produce flows? Did you check if you have valid licenses and not running in demo mode?

[lucaderi]

As of the topic error message did you create the following topics

static const char *message_topics[] = {
"flow", "event", "counter", "template", "option",
"hello", "listening-ports", "snmp-ifaces", "message", NULL};

?

Can you share a complete ntopng startup log?

[atemix]

Hi @lucaderi

Can you please check if nprobe in injecting flows into kafka and ntopng is unable to read them, or if the problem is that in your case nprobe does not produce flows?

Nprobe exports flows to kafka, Ntopng reads them. But only core fuctionality works. Flows apeears on live, and that's all.

Did you check if you have valid licenses and not running in demo mode?

Valid.

As of the topic error message did you create the following topics?

Nprobe automatically created all 8 topics after start, but errors in Ntopng reading kafka.

Can you share a complete ntopng startup log?

Unfortunately no, deployment reconfigured to use ZMQ, no kafka at now.