JVB can't connect with clients? What's the interconnection schema?

[alexander-potemkin]

I would much appreciate an assistance on understanding how JVB is supposed to connect clients from inside the Docker - I've spend quite a few days trying to figure that out, but with no luck.

my docker-compose.yaml - is from repository, my .env is provided below. When the meeting starts, clients can't hear each other - console log complains about ICE connection time-out, jvb container has the following error message repeating:

JVB 2024-10-29 16:29:22.418 INFO: [20] [confId=898c1a0acd50dd79 [email protected] meeting_id=5593d192 epId=f1ebfb4d stats_id=Jamarcus-l33 local_ufrag=6j0as1ibceabne ufrag=6j0as1ibceabne] Agent.triggerCheck#1727: Add peer CandidatePair with new reflexive address to checkList: CandidatePair (State=Frozen Priority=7961835276064522239):
	LocalCandidate=candidate:1 1 udp 2130706431 172.18.0.4 10000 typ host
	RemoteCandidate=candidate:10007 1 udp 1853759231 xx.xx.xx.xx 50446 typ prflx

My TURN & STUN servers pass icetest.info test, so I believe they are Ok.
But I can't figure out what's going on (and wrong) there from the message above.

.env:

# shellcheck disable=SC2034

# Directory where all configuration will be stored
CONFIG=~/.jitsi-meet-cfg

# Exposed HTTP port
HTTP_PORT=8000

# System time zone
TZ=Europe/Berlin

# Public URL for the web service (required)
PUBLIC_URL=https://<stripped>
JVB_STUN_SERVERS=<stripped>:3478
STUN_HOST=<stripped>
STUN_PORT=3478
TURN_HOST=<stripped>
TURN_PORT=5349
TURN_TRANSPORT=tcp
TURN_CREDENTIALS=<stripped>
TURNS_HOST=<stripped>
TURNS_PORT=5349
JVB_ADVERTISE_IPS=<PUBLIC_IP - stripped>
JVB_ADVERTISE_PRIVATE_CANDIDATES=false
ENABLE_P2P=false
ENABLE_COLIBRI_WEBSOCKET=true

ENABLE_JAAS_COMPONENTS=0
ENABLE_LETSENCRYPT=0

JICOFO_AUTH_PASSWORD=<stripped>
JVB_AUTH_PASSWORD=<stripped>
JIGASI_XMPP_PASSWORD=<stripped>
JIBRI_RECORDER_PASSWORD=<stripped>
JIBRI_XMPP_PASSWORD=<stripped>
RESTART_POLICY=unless-stopped

Any help would be much much appreciated!

[damencho]

JVB_ADVERTISE_IPS=<PUBLIC_IP - stripped>

Have you done port forwarding of port UDP 10000 from this public address to the machine hosting jvb?

Do you see any errors in the js console logs?

[alexander-potemkin]

@damencho , thank you for prompt responce!

Have you done port forwarding of port UDP 10000 from this public address to the machine hosting jvb?

Yes, sure - there is a direct UDP forward from PUBLIC-IP:10000 to the jvb-docker.

Do you see any errors in the js console logs?

Nop. I used to (related to ICE time-out) - before I exactly did a port forwarding from the public ip to jvb-docker. But not a single error on the browser side. I've managed to catch error from jvb docker.

[damencho]

Have you checked the js console now? Is the forwarding working in both directions? Maybe upload jvb logs.

[alexander-potemkin]

Have you checked the js console now?

Nothing in the console now... at least - not in terms of clearly visible errors.

Is the forwarding working in both directions?

I guess it should be... how can I double check?

Maybe upload jvb logs.

Sure, thank you! Please, find attached.
jvb.log
jitsi-console.log

[damencho]

Suspicious ICE connectivity failure. Checks failed but the remote end was able to reach us.

Seems clients can send packets to jvb but jvb is not able to respond, check your network.

[alexander-potemkin]

Suspicious ICE connectivity failure. Checks failed but the remote end was able to reach us.

Seems clients can send packets to jvb but jvb is not able to respond, check your network.

That's exactly what puzzles me - connectivity from jvb to outside world is not limited in any way:

$ docker exec -it ubuntu-jvb-1 "/bin/bash"

root@0c7fa5c824ad:/# ping 1.1.1.1
PING 1.1.1.1 (1.1.1.1): 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=56 time=5.923 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=56 time=6.612 ms
^C--- 1.1.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 5.923/6.268/6.612/0.345 ms

root@0c7fa5c824ad:/# nc -vv google.com 443
DNS fwd/rev mismatch: google.com != fra16s46-in-f14.1e100.net
google.com [216.58.212.142] 443 (https) open

And what kind of 'Checks failed' mentioned in the error message? How can I reproduce them to find the one that is failing?

[damencho]

Is it using the same route as the incoming packets?

And what kind of 'Checks failed' mentioned in the error message? How can I reproduce them to find the one that is failing?

You can go into the container and stop the jvb process, use a nc clients from a machine in internet and send a UDP packet to port 10000 to the public address of jvb, you should receive that packet in the container, now use the public address and port from where this packet is received to send a packet from jvb container and that should reach the client.
That second packet is not reaching the client, most probably is getting another route different from the incoming and the public IP used for the outgoing packet is different.

Here is explained how to test the incoming, you can do the same and for outgoing to check the public address used for the outgoing:
https://community.jitsi.org/t/tip-how-to-check-udp-10000-connectivity/73031

[alexander-potemkin]

Thank you! I don't, however, believe that the route from the server to the client - browser - is possible: the browser is running on the machine within a private network of my ISP - how can public facing server access IP from the private network (192.168....)?

[damencho]

Using the back route this is a response to a received packet, that's why the route of coming back need to be the same. This is called NAT latching. https://datatracker.ietf.org/doc/html/rfc7362
This is how jvb works if you open a meeting with 3 people, uses the port and address from where the packet is received.

[alexander-potemkin]

@damencho , thank you so much for your prompt responces and help!

Based on your input I rolled-out a separate server with public IP and everything worked just fine with my config. Setup I was trying to spin out was to launch jitsi from within the virtual machine, passing ports 10000/udp and 80/tcp via socat and caddy accordingly.

Something seems like went wrong exactly in the part of 10000/udp port forward and I'm still trying figure out what exactly. The very same setup works fine with WireGuard, for example, and many other TCP based apps, but here something seems to work different and I can't figure out what exactly.

If you have any kind of clue - I would much appreciate it! If not - I intend to keep trying to resolve that, shall I be successful - I will certainly return back here with the result, or some intermediate thoughts.

[damencho]

The common problem with stuff like that is that it changes the source address of the packages. So the public address and port of the client are lost and jvb is not able to send back media to the client.

[damencho]

Reading now for Socat, it does exactly that.
https://serverfault.com/questions/697939/socat-udp-packet-forwarding-with-source-address

Socat is a proxy, which means that it makes connections to a port/service/etc on behalf of something else. This means that from the perspective of your service, the connection originates with socat. There is in general no easy way to work around this behavior.

[alexander-potemkin]

@damencho , thank you again for all the help - it really helped me to figure out what could go wrong here and hence find a working solution.

For the traffic mirroring to work socat command line has to be adjusted in a way that I didn't encountered while searching for a ready to use solutions: UDP4-LISTEN:10000,reuseaddr,fork UDP4:jitsi-vm:10000 seems to be working - I will keep testing it, but for now it seems to be just fine.

[alexander-potemkin]

I'm sorry if that's a bit off-topic, but for me it feels connected: is there a need for TURN server, at all, given that JVB actually do the same, but on a different level / better?

[damencho]

The turnserver is needed in two cases. When a participant cannot use UDP, it will fallback to using TCP for the media via the turnserver. And the second one is to offload traffic from jvb in p2p calls when participants are not able to establish a direct connection, they need a relay - jvb and turnserver are such relays, if turn is not available the jvb will be used, but this only in case when direct p2p does not succeed.

[alexander-potemkin]

Just wanted to confirm that UDP4-LISTEN:10000,reuseaddr,fork UDP4:jitsi-vm:10000 works pretty well for me for a bit more than a month now - so it's something I could recommend, I believe.

[alexander-potemkin]

The turnserver is needed in two cases. When a participant cannot use UDP, it will fallback to using TCP for the media via the turnserver. And the second one is to offload traffic from jvb in p2p calls when participants are not able to establish a direct connection, they need a relay - jvb and turnserver are such relays, if turn is not available the jvb will be used, but this only in case when direct p2p does not succeed.

I'm sorry - I've only thank you in my head, but not in reality - so thank you very much!

I would like to test if or how it will work without turnserver - to make it all go via UDP and via jvb, even for a 1:1 calls, as I control the environment and can be pretty much sure that nothing prevents UDP from working.

If that's not too much to ask, could you please, advice - how I may achieve that with Docker with environment variables?