Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ntopng - traffic graph & stats only update once per minute, and on nprobe exit. #8448

Open
apezio opened this issue Jun 13, 2024 · 5 comments

Comments

@apezio
Copy link

apezio commented Jun 13, 2024

nprobe is only sending netflow-lite flow data to ntopng once per minute. Is this a bug? It seems unusable like this. The hosts list and flow list are typically empty by the time the data updates and the traffig graph in ntopng just shows a corresponding spike, once per minute but should show a steady 5Mbps (or whatever) of traffic.

ntopng will also update if I control-c the nprobe process.

The ntopng interface itself updates about once per second, showing about 5 ZMQ Messages per second (447 bytes each).

Once every minute there is a burst of 30 ZMQ's with a length of about 3700 bytes. Within a second or two the ntopng updates its traffic counts and traffic graph.

My router is sending a constant stream of flow information to nprobe (multiple pps).

I am running:

nprobe -i none -b 1 -s 5 -W --nflite 2055:1 -n 127.0.0.1:5556 --zmq "tcp://*:5556" -e 0
ntopng -i tcp://127.0.0.1:5556
(same host)

also I have tried about 200 other command line options.... nothing has "fixed" the issue. I also tried nprobe 6.1.x

Version: 6.0.240612 [Enterprise/Professional build]
GIT rev: 6.0-stable:98954d1c8729c3fa8d2221dd277e1d38dceb761d:20240612
Pro rev: r6175
Built on: CentOS Linux release 7.9.2009 (Core)
System Id: LA2C62ABCB205A206--UA2C62ABC9F24C390--OL
Platform: x86_64
Edition: Community
License Type: Time-Limited [Empty license file]
Validity: Until Thu Jun 13 02:33:29 2024

router#show netflow-lite exporter check
Netflow-lite Exporter check:
Network Protocol Configuration:
Destination IP address:
Source IP Address:
VRF label: none
DSCP: 0x0
TTL: 254
COS: 0
Transport Protocol Configuration:
Transport Protocol: UDP
Source Port: 52316
Destination Port: 2055
Destination Ports to Load-share: 1
Export Protocol Configuration:
Export Protocol: netflow-v9
Template data timeout: 60
Options sampler-table timeout: 60
Options interface-table timeout: 1800
Exporter Statistics:
Packets Exported: 863673

Just spent 10 hours on this - Any help would be appreciated!

nprobe stats output:

[root@ntop ~]# nprobe -i none -b 1 -s 5 -W --nflite 2055:1 -n 127.0.0.1:5556 --zmq "tcp://:5556" -e 0
13/Jun/2024 02:32:34 [plugin.c:178] No plugins found in ./plugins
13/Jun/2024 02:32:34 [plugin.c:186] Loading 23 plugins [.so] from /usr/lib/nprobe/plugins
13/Jun/2024 02:32:34 [nprobe.c:6499] WARNING: The minimum snaplen is 64
13/Jun/2024 02:32:34 [nprobe.c:7749] IMPORTANT: Enabling NflitePlugin will also enable IP address forging, thus
13/Jun/2024 02:32:34 [nprobe.c:7750] IMPORTANT: flows appear as they were sent from the NflitePlugin-enabled switch
13/Jun/2024 02:32:34 [nprobe.c:4848] Exporting flows towards 127.0.0.1:5556 using UDP
13/Jun/2024 02:32:34 [nprobe.c:7149] WARNING: IMPORTANT
13/Jun/2024 02:32:34 [nprobe.c:7150] WARNING: IMPORTANT --zmq tcp://x.x.x.x:yyyy is deprecated
13/Jun/2024 02:32:34 [nprobe.c:7151] WARNING: IMPORTANT and it has been replaced with
13/Jun/2024 02:32:34 [nprobe.c:7152] WARNING: IMPORTANT --ntopng zmq://x.x.x.x:yyyy
13/Jun/2024 02:32:34 [nprobe.c:7153] WARNING: IMPORTANT Please update your configuration
13/Jun/2024 02:32:34 [nprobe.c:7154] WARNING: IMPORTANT
13/Jun/2024 02:32:34 [nprobe.c:5387] WARNING: Invalid license (/etc/nprobe.license) [Missing license file]
13/Jun/2024 02:32:34 [nprobe.c:5397] WARNING: ******************************************
13/Jun/2024 02:32:34 [nprobe.c:5398] WARNING: ** **
13/Jun/2024 02:32:34 [nprobe.c:5399] WARNING: ** Switching to DEMO MODE **
13/Jun/2024 02:32:34 [nprobe.c:5400] WARNING: ** - Missing license file **
13/Jun/2024 02:32:34 [nprobe.c:5401] WARNING: ** **
13/Jun/2024 02:32:34 [nprobe.c:5403] WARNING: ** Purchase your license at **
13/Jun/2024 02:32:34 [nprobe.c:5404] WARNING: ** https://shop.ntop.org/ **
13/Jun/2024 02:32:34 [nprobe.c:5405] WARNING: ** **
13/Jun/2024 02:32:34 [nprobe.c:5407] WARNING: ******************************************
13/Jun/2024 02:32:34 [nprobe.c:7886] WARNING: The output interfaceId is set to 0: did you forget to use -Q perhaps ?
13/Jun/2024 02:32:34 [nprobe.c:7889] WARNING: The input interfaceId is set to 0: did you forget to use -u perhaps ?
13/Jun/2024 02:32:34 [nprobe.c:7914] Using ZMQ sourceId 1761666228
13/Jun/2024 02:32:34 [nprobe.c:7988] Welcome to nProbe v.10.4.240612 for x86_64-unknown-linux-gnu with native PF_RING acceleration
13/Jun/2024 02:32:34 [nprobe.c:8010] Pro Edition running on CentOS Linux release 7.9.2009 (Core)
13/Jun/2024 02:32:34 [nprobe.c:8018] Current limits [4 ZMQ exporters][4 collector devices]
13/Jun/2024 02:32:34 [nprobe.c:8029] SystemId: LA2C62ABCB205A206--UA2C62ABC9F24C390--OL
13/Jun/2024 02:32:34 [nprobe.c:8122] Sample rate [packet: 1][flow collection/export: 1/1]
13/Jun/2024 02:32:34 [nprobe.c:11490] WARNING: ***************************************************************
13/Jun/2024 02:32:34 [nprobe.c:11491] WARNING: * NOTE: This is a DEMO version limited to: *
13/Jun/2024 02:32:34 [nprobe.c:11492] WARNING: * - flows export: 5000 (live), 512 (pcap). *
13/Jun/2024 02:32:34 [nprobe.c:11493] WARNING: * - 300 seconds. *
13/Jun/2024 02:32:34 [nprobe.c:11494] WARNING: ***************************************************************
13/Jun/2024 02:32:34 [exportPlugin.c:664] WARNING: Kafka support requires nprobe Enterprise M or better: disabled
13/Jun/2024 02:32:34 [nflitePlugin.c:909] ERROR: Flow collector port 2055/IPv6 already in use ? [Address family not supported by protocol/97]: disabling collection over IPv6
13/Jun/2024 02:32:34 [nflitePlugin.c:933] [NFLite] Listening on port range 2055-2055 (1)
13/Jun/2024 02:32:34 [nprobe.c:10217] Using template %IN_SRC_MAC %OUT_DST_MAC %INPUT_SNMP %OUTPUT_SNMP %SRC_VLAN %IPV4_SRC_ADDR %IPV4_DST_ADDR %L4_SRC_PORT %L4_DST_PORT %IPV6_SRC_ADDR %IPV6_DST_ADDR %IP_PROTOCOL_VERSION %PROTOCOL %L7_PROTO %L7_CONFIDENCE %IN_BYTES %IN_PKTS %OUT_BYTES %OUT_PKTS %FIRST_SWITCHED %LAST_SWITCHED %CLIENT_TCP_FLAGS %SERVER_TCP_FLAGS %L7_PROTO_RISK %L7_RISK_SCORE %EXPORTER_IPV4_ADDRESS %DIRECTION %SAMPLING_INTERVAL %TOTAL_FLOWS_EXP %NPROBE_IPV4_ADDRESS %NPROBE_INSTANCE_NAME %CLIENT_NW_LATENCY_MS %SERVER_NW_LATENCY_MS %APPL_LATENCY_MS %TCP_WIN_MAX_IN %TCP_WIN_MAX_OUT %OOORDER_IN_PKTS %OOORDER_OUT_PKTS %RETRANSMITTED_IN_PKTS %RETRANSMITTED_OUT_PKTS %SRC_FRAGMENTS %DST_FRAGMENTS %L7_INFO %DNS_QUERY %DNS_QUERY_TYPE %DNS_RET_CODE %HTTP_URL %HTTP_SITE %HTTP_METHOD %HTTP_RET_CODE %TLS_SERVER_NAME %BITTORRENT_HASH %SRC_TOS %DST_TOS %HTTP_USER_AGENT %L7_RISK_INFO
13/Jun/2024 02:32:34 [nprobe.c:10219] Using NetFlow Packet Payload Len: 1472
13/Jun/2024 02:32:34 [template.c:3570] WARNING: Unable to locate template 'NPROBE_IPV6_ADDRESS': Pro version does not include plugins.
13/Jun/2024 02:32:34 [template.c:3570] WARNING: Unable to locate template 'NPROBE_IPV6_ADDRESS': Pro version does not include plugins.
13/Jun/2024 02:32:34 [plugin.c:1205] 3 plugin(s) enabled
13/Jun/2024 02:32:34 [nprobe.c:10575] Skipping plugin Netflow-Lite Plugin: no IEs defined
13/Jun/2024 02:32:34 [nprobe.c:10760] Each flow is 776 bytes long
13/Jun/2024 02:32:34 [nprobe.c:10761] The # flows per packet has been set to 1
13/Jun/2024 02:32:34 [nprobe.c:10764] IP TOS is ignored
13/Jun/2024 02:32:34 [nprobe.c:11566] Flow export type (-T): bidirectional flows
13/Jun/2024 02:32:34 [nprobe.c:11774] Flows ASs will not be computed (no GeoDB files loaded with --as-list)
13/Jun/2024 02:32:34 [nprobe.c:11806] Flows will be exported in NetFlow 9 format
13/Jun/2024 02:32:34 [nprobe.c:11852] Learning the public IP address.. Disable it with --disable-startup-checks
13/Jun/2024 02:32:34 [util.c:6457] Initializing ZMQ as server
13/Jun/2024 02:32:34 [util.c:6494] Successfully created ZMQ endpoint tcp://
:5556 with sourceId: 1761666228
13/Jun/2024 02:32:34 [nprobe.c:12044] Not capturing packet from interface (collector mode)
13/Jun/2024 02:32:34 [plugin.c:1000] Enabling plugin DNS/LLMNR Protocol
13/Jun/2024 02:32:34 [plugin.c:1000] Enabling plugin HTTP Protocol
13/Jun/2024 02:32:34 [plugin.c:1000] Enabling plugin Netflow-Lite Plugin
13/Jun/2024 02:32:34 [export.c:483] Using TLV as serialization format
13/Jun/2024 02:32:34 [nprobe.c:12340] nProbe started successfully
13/Jun/2024 02:32:35 [nprobe.c:4279] ---------------------------------
13/Jun/2024 02:32:35 [nprobe.c:4282] Average traffic: [0.00 pps][All Traffic 0 b/sec][IP Traffic 0 b/sec][ratio -nan]
13/Jun/2024 02:32:35 [nprobe.c:4290] Current traffic: [0.00 pps][0 b/sec]
13/Jun/2024 02:32:35 [nprobe.c:4298] L7 Proto Diff Total
13/Jun/2024 02:32:35 [nprobe.c:4322] Flows exports (including drops) [0 flows][avg: 0.0 flows/sec][latest 1 sec avg: 0.0 flows/sec]
13/Jun/2024 02:32:35 [nprobe.c:4339] Flow drops [export queue full: 0]
13/Jun/2024 02:32:35 [nprobe.c:4342] Packet drops [too many flow buckets: 0]
13/Jun/2024 02:32:35 [nprobe.c:4345] Flow Buckets [active: 0][allocated: 0][toBeExported: 0]
13/Jun/2024 02:32:35 [nprobe.c:4349] Export Queue [current: 0][max: 512000][fill level: 0.0%]
13/Jun/2024 02:32:35 [nprobe.c:4380] ZMQ Export [1 exporters][0 flows][total avg: 0 b/sec (0 b/sec traffic)][0.0 bytes/flow][latest 1 sec avg: 0 b/sec]
13/Jun/2024 02:32:35 [nflitePlugin.c:991] [NFLite] [# Template Pkts Rcvd: 0][# Flows with Unknown Templates: 36]
13/Jun/2024 02:32:35 [nflitePlugin.c:994] [NFLite] [# Templates Defined: 0][# Flows Rcvd: 36][# Data Flows: 0][# Bad Flows: 0]
13/Jun/2024 02:32:35 [nflitePlugin.c:999] [NFLite] [# Flow Packets Lost: 0][Flow Sequence: 1097186-1097221 (35)][# Flow Rcvd: 35]
13/Jun/2024 02:32:35 [nprobe.c:4421] Collector Threads: [36 pkts]
13/Jun/2024 02:32:35 [nprobe.c:4099] Processed packets: 0 (max bucket search: 0)
13/Jun/2024 02:32:35 [nprobe.c:4080] Fragment queue length: 0
13/Jun/2024 02:32:35 [nprobe.c:4130] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
13/Jun/2024 02:32:35 [nprobe.c:4142] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %]
13/Jun/2024 02:32:35 [nprobe.c:4148] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
13/Jun/2024 02:33:35 [nprobe.c:4279] ---------------------------------
13/Jun/2024 02:33:35 [nprobe.c:4282] Average traffic: [1.37 K pps][All Traffic 440.24 Kb/sec][IP Traffic 144.31 Kb/sec][ratio 0.35]
13/Jun/2024 02:33:35 [nprobe.c:4290] Current traffic: [1.12 K pps][359.53 Kb/sec]
13/Jun/2024 02:33:35 [nprobe.c:4298] L7 Proto Diff Total
13/Jun/2024 02:33:35 [nprobe.c:4312] Unknown/0 883.91 KB 883.91 KB
13/Jun/2024 02:33:35 [nprobe.c:4322] Flows exports (including drops) [0 flows][avg: 0.0 flows/sec][latest 60 sec avg: 0.0 flows/sec]
13/Jun/2024 02:33:35 [nprobe.c:4339] Flow drops [export queue full: 0]
13/Jun/2024 02:33:35 [nprobe.c:4342] Packet drops [too many flow buckets: 0]
13/Jun/2024 02:33:35 [nprobe.c:4345] Flow Buckets [active: 387][allocated: 387][toBeExported: 0]
13/Jun/2024 02:33:35 [nprobe.c:4349] Export Queue [current: 0][max: 512000][fill level: 0.0%]
13/Jun/2024 02:33:35 [nprobe.c:4380] ZMQ Export [1 exporters][0 flows][total avg: 3.71 Kb/sec (665.56 Kb/sec traffic)][0.0 bytes/flow][latest 60 sec avg: 3.03 Kb/sec]
13/Jun/2024 02:33:35 [nflitePlugin.c:991] [NFLite] [# Template Pkts Rcvd: 2][# Flows with Unknown Templates: 550]
13/Jun/2024 02:33:35 [nflitePlugin.c:994] [NFLite] [# Templates Defined: 2][# Flows Rcvd: 2656][# Data Flows: 2103][# Bad Flows: 0]
13/Jun/2024 02:33:35 [nflitePlugin.c:999] [NFLite] [# Flow Packets Lost: 0][Flow Sequence: 1097186-1099841 (2655)][# Flow Rcvd: 2655]
13/Jun/2024 02:33:35 [nprobe.c:4421] Collector Threads: [2656 pkts]
13/Jun/2024 02:33:35 [nprobe.c:4099] Processed packets: 67296 (max bucket search: 1)
13/Jun/2024 02:33:35 [nprobe.c:4080] Fragment queue length: 0
13/Jun/2024 02:33:35 [nprobe.c:4130] Flow export stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
13/Jun/2024 02:33:35 [nprobe.c:4142] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %]
13/Jun/2024 02:33:35 [nprobe.c:4148] Total flow stats: [0 bytes/0 pkts][0 flows/0 pkts sent]
13/Jun/2024 02:34:35 [nprobe.c:4279] ---------------------------------
13/Jun/2024 02:34:35 [nprobe.c:4282] Average traffic: [1.36 K pps][All Traffic 455.38 Kb/sec][IP Traffic 162.12 Kb/sec][ratio 0.38]
13/Jun/2024 02:34:35 [nprobe.c:4290] Current traffic: [1.35 K pps][467.74 Kb/sec]
13/Jun/2024 02:34:35 [nprobe.c:4298] L7 Proto Diff Total
13/Jun/2024 02:34:35 [nprobe.c:4312] Unknown/0 1.29 MB 2.16 MB
13/Jun/2024 02:34:35 [nprobe.c:4331] Flows exports (including drops) [2 flows][avg: 0.0 flows/sec][latest 60 sec avg: 0.0 flows/sec]
13/Jun/2024 02:34:35 [nprobe.c:4339] Flow drops [export queue full: 0]
13/Jun/2024 02:34:35 [nprobe.c:4342] Packet drops [too many flow buckets: 0]
13/Jun/2024 02:34:35 [nprobe.c:4345] Flow Buckets [active: 592][allocated: 592][toBeExported: 0]
13/Jun/2024 02:34:35 [nprobe.c:4349] Export Queue [current: 0][max: 512000][fill level: 0.0%]
13/Jun/2024 02:34:35 [nprobe.c:4380] ZMQ Export [1 exporters][2 flows][total avg: 3.71 Kb/sec (678.67 Kb/sec traffic)][606.5 bytes/flow][latest 60 sec avg: 3.70 Kb/sec]
13/Jun/2024 02:34:35 [nflitePlugin.c:991] [NFLite] [# Template Pkts Rcvd: 4][# Flows with Unknown Templates: 550]
13/Jun/2024 02:34:35 [nflitePlugin.c:994] [NFLite] [# Templates Defined: 2][# Flows Rcvd: 5192][# Data Flows: 4636][# Bad Flows: 0]
13/Jun/2024 02:34:35 [nflitePlugin.c:999] [NFLite] [# Flow Packets Lost: 0][Flow Sequence: 1097186-1102377 (5191)][# Flow Rcvd: 5191]
13/Jun/2024 02:34:35 [nprobe.c:4421] Collector Threads: [5192 pkts]
13/Jun/2024 02:34:35 [nprobe.c:4099] Processed packets: 148352 (max bucket search: 1)
13/Jun/2024 02:34:35 [nprobe.c:4080] Fragment queue length: 0
13/Jun/2024 02:34:35 [nprobe.c:4130] Flow export stats: [9856 bytes/128 pkts][2 flows/3 pkts sent]
13/Jun/2024 02:34:35 [nprobe.c:4142] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %]
13/Jun/2024 02:34:35 [nprobe.c:4148] Total flow stats: [9856 bytes/128 pkts][2 flows/3 pkts sent]
13/Jun/2024 02:35:35 [nprobe.c:4279] ---------------------------------
13/Jun/2024 02:35:35 [nprobe.c:4282] Average traffic: [1.25 K pps][All Traffic 425.88 Kb/sec][IP Traffic 155.72 Kb/sec][ratio 0.39]
13/Jun/2024 02:35:35 [nprobe.c:4290] Current traffic: [1.06 K pps][372.30 Kb/sec]
13/Jun/2024 02:35:35 [nprobe.c:4298] L7 Proto Diff Total
13/Jun/2024 02:35:35 [nprobe.c:4312] Unknown/0 1.06 MB 3.21 MB
13/Jun/2024 02:35:35 [nprobe.c:4331] Flows exports (including drops) [303 flows][avg: 5.1 flows/sec][latest 60 sec avg: 5.0 flows/sec]
13/Jun/2024 02:35:35 [nprobe.c:4339] Flow drops [export queue full: 0]
13/Jun/2024 02:35:35 [nprobe.c:4342] Packet drops [too many flow buckets: 0]
13/Jun/2024 02:35:35 [nprobe.c:4345] Flow Buckets [active: 487][allocated: 487][toBeExported: 0]
13/Jun/2024 02:35:35 [nprobe.c:4349] Export Queue [current: 0][max: 512000][fill level: 0.0%]
13/Jun/2024 02:35:35 [nprobe.c:4380] ZMQ Export [1 exporters][303 flows][total avg: 12.14 Kb/sec (631.60 Kb/sec traffic)][605.2 bytes/flow][latest 60 sec avg: 27.47 Kb/sec]
13/Jun/2024 02:35:35 [nflitePlugin.c:991] [NFLite] [# Template Pkts Rcvd: 6][# Flows with Unknown Templates: 550]
13/Jun/2024 02:35:35 [nflitePlugin.c:994] [NFLite] [# Templates Defined: 2][# Flows Rcvd: 7181][# Data Flows: 6622][# Bad Flows: 0]
13/Jun/2024 02:35:35 [nflitePlugin.c:999] [NFLite] [# Flow Packets Lost: 0][Flow Sequence: 1097186-1104366 (7180)][# Flow Rcvd: 7180]
13/Jun/2024 02:35:35 [nprobe.c:4421] Collector Threads: [7181 pkts]
13/Jun/2024 02:35:35 [nprobe.c:4099] Processed packets: 211904 (max bucket search: 2)
13/Jun/2024 02:35:35 [nprobe.c:4080] Fragment queue length: 0
13/Jun/2024 02:35:35 [nprobe.c:4130] Flow export stats: [11529024 bytes/19552 pkts][303 flows/331 pkts sent]
13/Jun/2024 02:35:35 [nprobe.c:4142] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %]
13/Jun/2024 02:35:35 [nprobe.c:4148] Total flow stats: [11529024 bytes/19552 pkts][303 flows/331 pkts sent]
13/Jun/2024 02:36:35 [nprobe.c:4279] ---------------------------------
13/Jun/2024 02:36:35 [nprobe.c:4282] Average traffic: [1.15 K pps][All Traffic 389.05 Kb/sec][IP Traffic 141.37 Kb/sec][ratio 0.39]
13/Jun/2024 02:36:35 [nprobe.c:4290] Current traffic: [855.00 pps][285.31 Kb/sec]
13/Jun/2024 02:36:35 [nprobe.c:4298] L7 Proto Diff Total
13/Jun/2024 02:36:35 [nprobe.c:4312] Unknown/0 757.19 KB 3.95 MB
13/Jun/2024 02:36:35 [nprobe.c:4331] Flows exports (including drops) [427 flows][avg: 7.1 flows/sec][latest 60 sec avg: 2.1 flows/sec]
13/Jun/2024 02:36:35 [nprobe.c:4339] Flow drops [export queue full: 0]
13/Jun/2024 02:36:35 [nprobe.c:4342] Packet drops [too many flow buckets: 0]
13/Jun/2024 02:36:35 [nprobe.c:4345] Flow Buckets [active: 520][allocated: 520][toBeExported: 0]
13/Jun/2024 02:36:35 [nprobe.c:4349] Export Queue [current: 0][max: 512000][fill level: 0.0%]
13/Jun/2024 02:36:35 [nprobe.c:4380] ZMQ Export [1 exporters][427 flows][total avg: 12.61 Kb/sec (577.64 Kb/sec traffic)][605.3 bytes/flow][latest 60 sec avg: 13.91 Kb/sec]
13/Jun/2024 02:36:35 [nflitePlugin.c:991] [NFLite] [# Template Pkts Rcvd: 8][# Flows with Unknown Templates: 550]
13/Jun/2024 02:36:35 [nflitePlugin.c:994] [NFLite] [# Templates Defined: 2][# Flows Rcvd: 8788][# Data Flows: 8226][# Bad Flows: 0]
13/Jun/2024 02:36:35 [nflitePlugin.c:999] [NFLite] [# Flow Packets Lost: 0][Flow Sequence: 1097186-1105973 (8787)][# Flow Rcvd: 8787]
13/Jun/2024 02:36:35 [nprobe.c:4421] Collector Threads: [8788 pkts]
13/Jun/2024 02:36:35 [nprobe.c:4099] Processed packets: 263232 (max bucket search: 1)
13/Jun/2024 02:36:35 [nprobe.c:4080] Fragment queue length: 0
13/Jun/2024 02:36:35 [nprobe.c:4130] Flow export stats: [65549664 bytes/137792 pkts][427 flows/466 pkts sent]
13/Jun/2024 02:36:35 [nprobe.c:4142] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %]
13/Jun/2024 02:36:35 [nprobe.c:4148] Total flow stats: [65549664 bytes/137792 pkts][427 flows/466 pkts sent]
13/Jun/2024 02:37:35 [nprobe.c:4279] ---------------------------------
13/Jun/2024 02:37:35 [nprobe.c:4282] Average traffic: [1.09 K pps][All Traffic 369.93 Kb/sec][IP Traffic 135.27 Kb/sec][ratio 0.39]
13/Jun/2024 02:37:35 [nprobe.c:4290] Current traffic: [858.00 pps][296.96 Kb/sec]
13/Jun/2024 02:37:35 [nprobe.c:4298] L7 Proto Diff Total
13/Jun/2024 02:37:35 [nprobe.c:4312] Unknown/0 839.72 KB 4.77 MB
13/Jun/2024 02:37:35 [nprobe.c:4331] Flows exports (including drops) [597 flows][avg: 9.9 flows/sec][latest 60 sec avg: 2.8 flows/sec]
13/Jun/2024 02:37:35 [nprobe.c:4339] Flow drops [export queue full: 0]
13/Jun/2024 02:37:35 [nprobe.c:4342] Packet drops [too many flow buckets: 0]
13/Jun/2024 02:37:35 [nprobe.c:4345] Flow Buckets [active: 539][allocated: 539][toBeExported: 0]
13/Jun/2024 02:37:35 [nprobe.c:4349] Export Queue [current: 0][max: 512000][fill level: 0.0%]
13/Jun/2024 02:37:35 [nprobe.c:4380] ZMQ Export [1 exporters][597 flows][total avg: 13.60 Kb/sec (548.61 Kb/sec traffic)][606.0 bytes/flow][latest 60 sec avg: 17.37 Kb/sec]
13/Jun/2024 02:37:35 [nflitePlugin.c:991] [NFLite] [# Template Pkts Rcvd: 10][# Flows with Unknown Templates: 550]
13/Jun/2024 02:37:35 [nflitePlugin.c:994] [NFLite] [# Templates Defined: 2][# Flows Rcvd: 10401][# Data Flows: 9836][# Bad Flows: 0]
13/Jun/2024 02:37:35 [nflitePlugin.c:999] [NFLite] [# Flow Packets Lost: 0][Flow Sequence: 1097186-1107586 (10400)][# Flow Rcvd: 10400]
13/Jun/2024 02:37:35 [nprobe.c:4421] Collector Threads: [10401 pkts]
13/Jun/2024 02:37:35 [nprobe.c:4099] Processed packets: 314752 (max bucket search: 1)
13/Jun/2024 02:37:35 [nprobe.c:4080] Fragment queue length: 0
13/Jun/2024 02:37:35 [nprobe.c:4130] Flow export stats: [80520768 bytes/160384 pkts][597 flows/652 pkts sent]
13/Jun/2024 02:37:35 [nprobe.c:4142] Flow export drop stats: [0 bytes/0 pkts][0 flows/0.00 %]
13/Jun/2024 02:37:35 [nprobe.c:4148] Total flow stats: [80520768 bytes/160384 pkts][597 flows/652 pkts sent]
13/Jun/2024 02:37:35 [export.c:532] ERROR: ***************************************************************************
13/Jun/2024 02:37:35 [export.c:533] ERROR: * NOTE: You have reached the max demo 597 flows export: no more exports *
13/Jun/2024 02:37:35 [export.c:535] ERROR: * NOTE: no additional flows will be exported by this nProbe instance *
13/Jun/2024 02:37:35 [export.c:536] ERROR: ***************************************************************************
13/Jun/2024 02:38:35 [nprobe.c:4279] ---------------------------------
13/Jun/2024 02:38:35 [nprobe.c:4282] Average traffic: [1.04 K pps][All Traffic 356.68 Kb/sec][IP Traffic 131.48 Kb/sec][ratio 0.40]
13/Jun/2024 02:38:35 [nprobe.c:4290] Current traffic: [833.00 pps][292.85 Kb/sec]
13/Jun/2024 02:38:35 [nprobe.c:4298] L7 Proto Diff Total
13/Jun/2024 02:38:35 [nprobe.c:4312] Unknown/0 849.28 KB 5.60 MB
13/Jun/2024 02:38:35 [nprobe.c:4331] Flows exports (including drops) [597 flows][avg: 9.9 flows/sec][latest 60 sec avg: 0.0 flows/sec]
13/Jun/2024 02:38:35 [nprobe.c:4339] Flow drops [export queue full: 0]
13/Jun/2024 02:38:35 [nprobe.c:4342] Packet drops [too many flow buckets: 0]
13/Jun/2024 02:38:35 [nprobe.c:4345] Flow Buckets [active: 527][allocated: 527][toBeExported: 0]
13/Jun/2024 02:38:35 [nprobe.c:4349] Export Queue [current: 0][max: 512000][fill level: 0.0%]
13/Jun/2024 02:38:35 [nprobe.c:4380] ZMQ Export [1 exporters][597 flows][total avg: 11.27 Kb/sec (528.16 Kb/sec traffic)][606.0 bytes/flow][latest 60 sec avg: 54 b/sec]
13/Jun/2024 02:38:35 [nflitePlugin.c:991] [NFLite] [# Template Pkts Rcvd: 12][# Flows with Unknown Templates: 550]
13/Jun/2024 02:38:35 [nflitePlugin.c:994] [NFLite] [# Templates Defined: 2][# Flows Rcvd: 11967][# Data Flows: 11399][# Bad Flows: 0]
13/Jun/2024 02:38:35 [nflitePlugin.c:999] [NFLite] [# Flow Packets Lost: 0][Flow Sequence: 1097186-1109152 (11966)][# Flow Rcvd: 11966]

int-traf

@apezio
Copy link
Author

apezio commented Jun 13, 2024

I have tried --zmq-disable-buffering. I synced the clocks on the cisco 4948e-f and the server.

I've been attempting to use the instructions from:

https://www.ntop.org/nprobe/say-hello-to-netflow-lite-nflite/
and
https://www.ntop.org/guides/ntopng/using_with_other_tools/nprobe.html
and
https://www.ntop.org/NetFlowLite/netflow-lite_webinar-cisco.pdf

A lot of conflicting information to be sure, but i tried just about every combo of options I could possibly imagine might have some effect.

@apezio
Copy link
Author

apezio commented Jun 13, 2024

FYI it works as expected when using a local nework interface (i.e. nprobe -i eth0).

@lucaderi
Copy link
Member

nflite is a clone of sFlow and it sends traffic as soon as the switch decides it's time to export. CAn you please check with wireshark how often nfLite exports you packet dumps?

@apezio
Copy link
Author

apezio commented Jun 19, 2024

Templates are exported every 60 seconds and flows are 1000 packets per second.

@apezio
Copy link
Author

apezio commented Jun 22, 2024

Here are some tcpdumps of the nflite UDP packets coming from the cisco 4948e.

nflite.pcap.gz
972nflite-pkts.pcap.gz

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants