Does F-stack queue packets until they reach a certain number before dispatching to the upper layer when sending & receiving tcp packets?

[harveyliuit]

Can anyone know please answer my questions? Thanks in advance!

[freak82]

My knowledge is from 1-2 years ago (things may have changed in the F-stack glue code) but I'll try to answer your question:

1. Receiving TCP packets

• the packets are received in bursts (group of packets) from the NIC and are immediately injected in the FreeBSD network stack
• if the packets are received in order you should be able to read the content from them as soon as you initiate reading from the corresponding socket
• if the packets are received out of order they will be placed in the receive buffer of the corresponding socket and remain there until the in-order sequence is received and then you'll be able to read them from the socket buffer.

2. Sending TCP data

• if the TCP congestion control (CC) algorithm detects congestion then the sent data will remain in the send buffer of the corresponding socket and no packets (mbufs) will be transferred to the lower layers until the CC algorithm allows this. Receiving some packet(s) from the remote or timing out may trigger some mbufs to be transferred to the lower layers.
• in the more common case you'll send data on the socket and the CC algorithm will allow this data to be immediately sent. In this case, the corresponding mbufs will be transferred to the lower layers and reach the F-stack glue code (the F-stack code "glues" the DPDK machinery and the NIC management with the FreeBSD stack).
• when reaching the F-stack glue code the mbufs will be either put into a memory array. The packets from this array will be either sent immediately if the array becomes full (the size of this array is MAX_PKT_BURST = 32), or sent on the next iteration of the F-stack main loop (if the pkt_tx_delay = 0) or sent every pkt_tx_delay microseconds if the latter is > 0. The pkt_tx_delay comes from the config file.
• So, if you care about latency of the sent packets you should set the pkt_tx_delay to 0 or very small number. If you care about throughput you should use some bigger value there or use the default one. Note that sending packets to the NIC in bursts is more effective than sending them one by one and that's the reason for the existing of this configuration option.

HTH.