Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unable to connect to the server instance (OpenWrt) #523

Closed
o-alquimista opened this issue Apr 28, 2024 · 16 comments
Closed

Unable to connect to the server instance (OpenWrt) #523

o-alquimista opened this issue Apr 28, 2024 · 16 comments

Comments

@o-alquimista
Copy link

o-alquimista commented Apr 28, 2024 •
edited
Loading

Client

System info:

  • Fedora 40 x86_64 laptop
  • firewalld active, default config
  • SELinux enforcing
  • udp2raw binary: udp2raw_amd64
$ sudo udp2raw_amd64 -c -l 0.0.0.0:55820 -r 203.0.113.100:4096 -k "password_removed" --raw-mode faketcp -a
[2024-04-28 16:49:08][INFO]argc=11 udp2raw_amd64 -c -l 0.0.0.0:55820 -r 203.0.113.100:4096 -k password_removed --raw-mode faketcp -a 
[2024-04-28 16:49:08][INFO]parsing address: 0.0.0.0:55820
[2024-04-28 16:49:08][INFO]its an ipv4 adress
[2024-04-28 16:49:08][INFO]ip_address is {0.0.0.0}, port is {55820}
[2024-04-28 16:49:08][INFO]parsing address: 203.0.113.100:4096
[2024-04-28 16:49:08][INFO]its an ipv4 adress
[2024-04-28 16:49:08][INFO]ip_address is {203.0.113.100}, port is {4096}
[2024-04-28 16:49:08][INFO]important variables: log_level=4:INFO raw_mode=faketcp cipher_mode=aes128cbc auth_mode=md5 key=password_removed local_addr=0.0.0.0:55820 remote_addr=203.0.113.100:4096 socket_buf_size=1048576 
[2024-04-28 16:49:08][WARN]you can run udp2raw with non-root account for better security. check README.md in repo for more info.
[2024-04-28 16:49:08][INFO]remote_ip=[203.0.113.100], make sure this is a vaild IP address
[2024-04-28 16:49:08][INFO]const_id:9bbac27b
[2024-04-28 16:49:08][INFO]run_command iptables -N udp2rawDwrW_9bbac27b_C0
[2024-04-28 16:49:08][INFO]run_command iptables -F udp2rawDwrW_9bbac27b_C0
[2024-04-28 16:49:08][INFO]run_command iptables -I udp2rawDwrW_9bbac27b_C0 -j DROP
[2024-04-28 16:49:08][INFO]run_command iptables -I INPUT -s 203.0.113.100 -p tcp -m tcp --sport 4096 -j udp2rawDwrW_9bbac27b_C0
[2024-04-28 16:49:08][WARN]auto added iptables rules
[2024-04-28 16:49:08][INFO]source_addr is now 192.168.0.110
[2024-04-28 16:49:08][INFO]using port 47991
[2024-04-28 16:49:08][INFO]state changed from client_idle to client_tcp_handshake
[2024-04-28 16:49:08][INFO](re)sent tcp syn
[2024-04-28 16:49:09][INFO](re)sent tcp syn
[2024-04-28 16:49:10][INFO](re)sent tcp syn
[2024-04-28 16:49:12][INFO](re)sent tcp syn
[2024-04-28 16:49:13][INFO](re)sent tcp syn

Replaced actual public address with 203.0.113.100

Server

System info:

/etc/config/firewall

(...)
config include 'Include_Udp2raw'          
        option type 'nftables'        
        option path '/etc/udp2raw.nft'
        option position 'chain-pre'               
        option chain 'input'

/etc/udp2raw.nft

tcp dport 4096 counter drop

/etc/init.d/udp2raw

#!/bin/sh /etc/rc.common

USE_PROCD=1
START=95
STOP=01

start_service() {
	procd_open_instance
	procd_set_param command udp2raw -s -l 0.0.0.0:4096 -r 127.0.0.1:55820 -k "password_removed" --raw-mode faketcp
	procd_set_param stdout 1
	procd_set_param stderr 1
	procd_close_instance
}
  • If I add an ACCEPT rule for port 4096 - instead of DROP, the client reaches the client_ready state - the connection appears to be successful. But then I cannot start Wireguard on the client because the port is already being used (by udp2raw).
  • If I don't add any rule for udp2raw, I get the same output I pasted in the client section above ([INFO](re)sent tcp syn echoing forever). It seems the DROP rules are not working as expected.
@wangyu-
Copy link
Owner

wangyu- commented Apr 29, 2024

If I add an ACCEPT rule for port 4096 - instead of DROP, the client reaches the client_ready state - the connection appears to be successful. But then I cannot start Wireguard on the client because the port is already being used (by udp2raw).

"Wireguard port being used" is not relevant to iptables or nftables rules or udp2raw can connect or not.

If it says port occupied there must be some problem in otherwhere.

Check your settings of wireguard, and see if you specific some source port to use. Remove the source port or change to another source port.

On the client side it's typically not necessary to specify a source port.

@o-alquimista
Copy link
Author

o-alquimista commented May 6, 2024 •
edited
Loading

I commented out the ListenPort in the wireguard client config, so it picks a random port on startup. After bringing wireguard up, I take note of the port number and pass it to the udp2raw client command: -l0.0.0.0:<random_port>. But sadly it's still not working.
It's not completing the handshake. It's like client and server are not agreeing on something. I even see "new packet from 127.0.0.1:55820".

I'm about to give up trying to run udp2raw on OpenWrt. I've used it successfully before on a Raspberry Pi. I just wonder if it's possible to keep wireguard on the router, and run udp2raw somewhere else in the LAN.

EDIT: I realized I should not use the ListenPort (random) in the udp2raw -l 0.0.0.0:<port> argument, but instead use the Endpoint (55820) port.

@o-alquimista
Copy link
Author

o-alquimista commented Jun 10, 2024 •
edited
Loading

I moved the udp2raw server to a system running Alpine Linux, and the problem persists.

[2024-06-10 13:27:04][INFO]argc=10 udp2raw -c -l 0.0.0.0:51820 -r PUBLIC_IPV4:4096 -k PASSWORD --raw-mode faketcp 
[2024-06-10 13:27:04][INFO]parsing address: 0.0.0.0:51820
[2024-06-10 13:27:04][INFO]its an ipv4 adress
[2024-06-10 13:27:04][INFO]ip_address is {0.0.0.0}, port is {51820}
[2024-06-10 13:27:04][INFO]parsing address: PUBLIC_IPV4:4096
[2024-06-10 13:27:04][INFO]its an ipv4 adress
[2024-06-10 13:27:04][INFO]ip_address is {PUBLIC_IPV4}, port is {4096}
[2024-06-10 13:27:04][INFO]important variables: log_level=4:INFO raw_mode=faketcp cipher_mode=aes128cbc auth_mode=md5 key=PASSWORD local_addr=0.0.0.0:51820 remote_addr=PUBLIC_IPV4:4096 socket_buf_size=1048576 
[2024-06-10 13:27:04][WARN]you can run udp2raw with non-root account for better security. check README.md in repo for more info.
[2024-06-10 13:27:04][INFO]remote_ip=[PUBLIC_IPV4], make sure this is a vaild IP address
[2024-06-10 13:27:04][INFO]const_id:7f9d8b1e
[2024-06-10 13:27:04][WARN] -a has not been set, make sure you have added the needed iptables rules manually
[2024-06-10 13:27:04][INFO]source_addr is now 192.168.1.108
[2024-06-10 13:27:04][INFO]using port 62321
[2024-06-10 13:27:04][INFO]state changed from client_idle to client_tcp_handshake
[2024-06-10 13:27:04][INFO](re)sent tcp syn
[2024-06-10 13:27:04][INFO]state changed from client_tcp_handshake to client_handshake1
[2024-06-10 13:27:04][INFO](re)sent handshake1
[2024-06-10 13:27:04][INFO]changed state from to client_handshake1 to client_handshake2,my_id is 364ef6d,oppsite id is 11b62c6
[2024-06-10 13:27:04][INFO](re)sent handshake2
[2024-06-10 13:27:04][INFO]changed state from to client_handshake2 to client_ready

At this point I turn on Wireguard...

[2024-06-10 13:27:12][INFO]new packet from 127.0.0.1:37727,conv_id=40053fc7
[2024-06-10 13:27:22][INFO]state back to client_idle from  client_ready bc of server-->client direction timeout
[2024-06-10 13:27:23][INFO]source_addr is now 192.168.2.4
[2024-06-10 13:27:23][INFO]using port 33196
[2024-06-10 13:27:23][INFO]state changed from client_idle to client_tcp_handshake
[2024-06-10 13:27:23][INFO](re)sent tcp syn
[2024-06-10 13:27:24][INFO](re)sent tcp syn
[2024-06-10 13:27:25][INFO](re)sent tcp syn
[2024-06-10 13:27:26][INFO](re)sent tcp syn

My steps:

  • Start udp2raw (it will reach client_ready)
  • Start wireguard (udp2raw will be back to client_idle)

Relevant log line:

[INFO]state back to client_idle from  client_ready bc of server-->client direction timeout

@wangyu- What does this mean?

Server info:

  • Alpine Linux (arm64) on Raspberry Pi 4B
  • nftables

@wangyu-
Copy link
Owner

wangyu- commented Jun 10, 2024 •
edited
Loading

[INFO]state back to client_idle from client_ready bc of server-->client direction timeout

It simply means the connection is broken.

Start udp2raw (it will reach client_ready)
Start wireguard (udp2raw will be back to client_idle)

Very likely, your wireguard is redirecting the route 0.0.0.0/0. And your udp2raw is hijacked by wireguard. Then it forms a traffic loop.

Have you added the route exception for udp2raw?

https://github.com/wangyu-/udp2raw/wiki/udp2raw-openvpn-config-guide

If you want to transparently redirect traffic by VPN, it's very important to add a route exception (on client side) so that the VPN won't hijack udp2raw's traffic. Since your VPN traffic goes throught udp2raw, if your VPN hijacks udp2raw's traffic then there will be a traffic loop and your udp2raw will lost connection

@wangyu-
Copy link
Owner

wangyu- commented Jun 10, 2024

try on client side:

ip route add YOUR_SERVER_IP via  YOUR_GATEWAY

@o-alquimista
Copy link
Author

Ah, I think I understand now. I have AllowedIPs set to unspecified on the client — tunneling everything through Wireguard, even udp2raw. The exception you suggested ensures udp2raw can talk "in the clear", outside of the Wireguard tunnel.

I just tested tunneling only local networks over Wireguard by setting AllowedIPs accordingly, and udp2raw works as expected — this is actually how I was running a few months ago, but now I need to tunnel internet traffic too. Will try the route exception now.

@o-alquimista
Copy link
Author

o-alquimista commented Jun 10, 2024 •
edited
Loading

Oops, accidentally posted my password just now. Ugh... will have to change it.

After adding the route, I'm still having trouble.

  1. udp2raw starts:
[2024-06-10 15:39:48][INFO]argc=10 udp2raw -c -l 0.0.0.0:51820 -r PUBLIC_IPV4:4096 -k PASSWORD --raw-mode faketcp 
[2024-06-10 15:39:48][INFO]parsing address: 0.0.0.0:51820
[2024-06-10 15:39:48][INFO]its an ipv4 adress
[2024-06-10 15:39:48][INFO]ip_address is {0.0.0.0}, port is {51820}
[2024-06-10 15:39:48][INFO]parsing address: PUBLIC_IPV4:4096
[2024-06-10 15:39:48][INFO]its an ipv4 adress
[2024-06-10 15:39:48][INFO]ip_address is {PUBLIC_IPV4}, port is {4096}
[2024-06-10 15:39:48][INFO]important variables: log_level=4:INFO raw_mode=faketcp cipher_mode=aes128cbc auth_mode=md5 key=PASSWORD local_addr=0.0.0.0:51820 remote_addr=PUBLIC_IPV4:4096 socket_buf_size=1048576 
[2024-06-10 15:39:48][WARN]you can run udp2raw with non-root account for better security. check README.md in repo for more info.
[2024-06-10 15:39:48][INFO]remote_ip=[PUBLIC_IPV4], make sure this is a vaild IP address
[2024-06-10 15:39:48][INFO]const_id:cc9eb324
[2024-06-10 15:39:48][WARN] -a has not been set, make sure you have added the needed iptables rules manually
[2024-06-10 15:39:48][INFO]source_addr is now 192.168.1.108
[2024-06-10 15:39:48][INFO]using port 54527
[2024-06-10 15:39:48][INFO]state changed from client_idle to client_tcp_handshake
[2024-06-10 15:39:48][INFO](re)sent tcp syn
[2024-06-10 15:39:48][INFO]state changed from client_tcp_handshake to client_handshake1
[2024-06-10 15:39:48][INFO](re)sent handshake1
[2024-06-10 15:39:48][INFO]changed state from to client_handshake1 to client_handshake2,my_id is 825de79d,oppsite id is d46a6b75
[2024-06-10 15:39:48][INFO](re)sent handshake2
[2024-06-10 15:39:48][INFO]changed state from to client_handshake2 to client_ready
  1. At this point, Wireguard interface goes up:
[2024-06-10 15:39:51][INFO]new packet from 127.0.0.1:38620,conv_id=e1512ec
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=1
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=2
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=3
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=4
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=5
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=6
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=7
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=8
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=9
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=10
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=11
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=12
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=13
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=14
[2024-06-10 15:39:51][WARN][PUBLIC_IPV4,4096]rst==1,cnt=15 >=max_rst_to_show, this log will be muted for current connection
[2024-06-10 15:40:01][INFO]state back to client_idle from  client_ready bc of server-->client direction timeout
[2024-06-10 15:40:01][INFO]source_addr is now 192.168.1.108
[2024-06-10 15:40:01][INFO]using port 26446
[2024-06-10 15:40:01][INFO]state changed from client_idle to client_tcp_handshake
[2024-06-10 15:40:01][INFO](re)sent tcp syn
[2024-06-10 15:40:01][INFO]state changed from client_tcp_handshake to client_handshake1
[2024-06-10 15:40:01][INFO](re)sent handshake1
[2024-06-10 15:40:01][INFO]changed state from to client_handshake1 to client_handshake2,my_id is 157e0bb1,oppsite id is d86a367e
[2024-06-10 15:40:01][INFO](re)sent handshake2
[2024-06-10 15:40:01][INFO]changed state from to client_handshake2 to client_ready
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=1
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=2
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=3
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=4
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=5
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=6
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=7
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=8
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=9
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=10
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=11
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=12
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=13
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=14
[2024-06-10 15:40:11][WARN][PUBLIC_IPV4,4096]rst==1,cnt=15 >=max_rst_to_show, this log will be muted for current connection
[2024-06-10 15:40:21][INFO]state back to client_idle from  client_ready bc of server-->client direction timeout
[2024-06-10 15:40:22][INFO]source_addr is now 192.168.1.108
[2024-06-10 15:40:22][INFO]using port 13299
[2024-06-10 15:40:22][INFO]state changed from client_idle to client_tcp_handshake
[2024-06-10 15:40:22][INFO](re)sent tcp syn
[2024-06-10 15:40:22][INFO]state changed from client_tcp_handshake to client_handshake1
[2024-06-10 15:40:22][INFO](re)sent handshake1
[2024-06-10 15:40:22][INFO]changed state from to client_handshake1 to client_handshake2,my_id is d1ffeec8,oppsite id is a552c38a
[2024-06-10 15:40:22][INFO](re)sent handshake2
[2024-06-10 15:40:22][INFO]changed state from to client_handshake2 to client_ready
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=1
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=2
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=3
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=4
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=5
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=6
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=7
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=8
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=9
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=10
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=11
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=12
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=13
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=14
[2024-06-10 15:40:25][WARN][PUBLIC_IPV4,4096]rst==1,cnt=15 >=max_rst_to_show, this log will be muted for current connection

EDIT: forgot to add command I used to add route.

ip route add PUBLIC_IPV4 via 192.168.1.1

192.168.1.1 is the default gateway in this client system.

EDIT 2:

Server logs (shortened):

Jun 10 16:03:01 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:03:01][INFO]now listening at 0.0.0.0:4096
Jun 10 16:04:32 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:32][INFO][192.168.1.1:50531]received syn,sent syn ack back
Jun 10 16:04:32 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:32][WARN][192.168.1.1,50531]rst==1,cnt=1
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:47][INFO][192.168.1.1:55565]received syn,sent syn ack back
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:47][WARN][192.168.1.1,55565]rst==1,cnt=1
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:47][INFO][192.168.1.1:55565]got packet from a new ip
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:47][INFO][192.168.1.1:55565]created new conn,state: server_handshake1,my_id is 2f7a21f2
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:47][INFO][192.168.1.1:55565]changed state to server_handshake1,my_id is 2f7a21f2
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:47][WARN][192.168.1.1,55565]rst==1,cnt=1
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:47][INFO][192.168.1.1:55565]received handshake oppsite_id:16927933  my_id:2f7a21f2
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:47][INFO][192.168.1.1:55565]oppsite const_id:854213fa 
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:04:47][INFO][192.168.1.1:55565]changed state to server_ready
Jun 10 16:04:47 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:47][WARN][192.168.1.1,55565]rst==1,cnt=2
Jun 10 16:04:48 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:48][WARN][192.168.1.1,55565]rst==1,cnt=3
Jun 10 16:04:49 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:49][WARN][192.168.1.1,55565]rst==1,cnt=4
Jun 10 16:04:49 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:49][WARN][192.168.1.1,55565]rst==1,cnt=5
Jun 10 16:04:50 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:50][WARN][192.168.1.1,55565]rst==1,cnt=6
Jun 10 16:04:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:51][WARN][192.168.1.1,55565]rst==1,cnt=7
Jun 10 16:04:52 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:52][WARN][192.168.1.1,55565]rst==1,cnt=8
Jun 10 16:04:53 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:53][WARN][192.168.1.1,55565]rst==1,cnt=9
Jun 10 16:04:53 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:53][WARN][192.168.1.1,55565]rst==1,cnt=10
Jun 10 16:04:54 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:54][WARN][192.168.1.1,55565]rst==1,cnt=11
Jun 10 16:04:55 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:55][WARN][192.168.1.1,55565]rst==1,cnt=12
Jun 10 16:04:56 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:56][WARN][192.168.1.1,55565]rst==1,cnt=13
Jun 10 16:04:57 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:57][WARN][192.168.1.1,55565]rst==1,cnt=14
Jun 10 16:04:57 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:04:57][WARN][192.168.1.1,55565]rst==1,cnt=15 >=max_rst_to_show, this log will be muted for current connection
Jun 10 16:07:26 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:26][INFO][192.168.1.1:55565]new conv conv_id=8175c854, assigned fd=10
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:38][INFO][192.168.1.1:18796]received syn,sent syn ack back
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:38][WARN][192.168.1.1,18796]rst==1,cnt=1
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:38][INFO][192.168.1.1:18796]got packet from a new ip
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:38][INFO][192.168.1.1:18796]created new conn,state: server_handshake1,my_id is d5b456f2
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:38][INFO][192.168.1.1:18796]changed state to server_handshake1,my_id is d5b456f2
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:38][WARN][192.168.1.1,18796]rst==1,cnt=1
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:38][INFO][192.168.1.1:18796]received handshake oppsite_id:53621ba9  my_id:d5b456f2
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:38][INFO][192.168.1.1:18796]oppsite const_id:854213fa 
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:38][INFO][192.168.1.1:18796]grabbed a connection
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:38][WARN][192.168.1.1,18796]rst==1,cnt=1
Jun 10 16:07:38 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:38][WARN][192.168.1.1,18796]rst==1,cnt=2
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=3
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=4
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=5
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=6
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=7
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=8
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=9
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=10
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=11
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=12
Jun 10 16:07:39 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:39][WARN][192.168.1.1,18796]rst==1,cnt=13
Jun 10 16:07:40 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:40][WARN][192.168.1.1,18796]rst==1,cnt=14
Jun 10 16:07:40 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:40][WARN][192.168.1.1,18796]rst==1,cnt=15 >=max_rst_to_show, this log will be muted for current connection
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:18796]inactive conn cleared 
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:64668]received syn,sent syn ack back
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:50][WARN][192.168.1.1,64668]rst==1,cnt=1
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:64668]got packet from a new ip
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:64668]created new conn,state: server_handshake1,my_id is 191d81c8
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:64668]changed state to server_handshake1,my_id is 191d81c8
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:50][WARN][192.168.1.1,64668]rst==1,cnt=1
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:64668]received handshake oppsite_id:1c08e048  my_id:191d81c8
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:64668]oppsite const_id:854213fa 
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[32m[2024-06-10 16:07:50][INFO][192.168.1.1:64668]grabbed a connection
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:50][WARN][192.168.1.1,64668]rst==1,cnt=1
Jun 10 16:07:50 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:50][WARN][192.168.1.1,64668]rst==1,cnt=2
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=3
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=4
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=5
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=6
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=7
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=8
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=9
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=10
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=11
Jun 10 16:07:51 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:51][WARN][192.168.1.1,64668]rst==1,cnt=12
Jun 10 16:07:52 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:52][WARN][192.168.1.1,64668]rst==1,cnt=13
Jun 10 16:07:52 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:52][WARN][192.168.1.1,64668]rst==1,cnt=14
Jun 10 16:07:52 oxygen daemon.info udp2raw: ^[[0m^[[33m[2024-06-10 16:07:52][WARN][192.168.1.1,64668]rst==1,cnt=15 >=max_rst_to_show, this log will be muted for current connection

The timestamps don't match with the client because I repeated the process later just to capture the server logs.

"oxygen" is the udp2raw server, but the Wireguard server is actually on another device (OpenWrt router, 192.168.1.1).

I also noticed firewall counters are zeroed on both server and client:

  • Server: tcp dport 4096 counter packets 0 bytes 0 drop comment "Drop udp2raw"
  • Client: ip saddr PUBLIC_IPV4 tcp sport 4096 counter packets 0 bytes 0 drop comment "Drop udp2raw"

@wangyu-
Copy link
Owner

wangyu- commented Jun 10, 2024

In your first post you are using -a at client side:

[2024-04-28 16:49:08][INFO]run_command iptables -I udp2rawDwrW_9bbac27b_C0 -j DROP
[2024-04-28 16:49:08][INFO]run_command iptables -I INPUT -s 203.0.113.100 -p tcp -m tcp --sport 4096 -j udp2rawDwrW_9bbac27b_C0
[2024-04-28 16:49:08][WARN]auto added iptables rules
[2024-04-28 16:49:08][INFO]source_addr is now 192.168.0.110
[2024-04-28 16:49:08][INFO]using port 47991

But in your lastest post you removed.

[2024-06-10 15:39:48][INFO]const_id:cc9eb324
[2024-06-10 15:39:48][WARN] -a has not been set, make sure you have added the needed iptables rules manually
[2024-06-10 15:39:48][INFO]source_addr is now 192.168.1.108

why do you make the change?

I tested udp2raw+wireguard with iptables, it worked well.

I don't use nftables. Cannot figure out what is going wrong for you.

I suggest you setup both side with iptables first, and make sure it works first. Then move to nftables and compare what is different.

@o-alquimista
Copy link
Author

In your first post you are using -a at client side:

[2024-04-28 16:49:08][INFO]run_command iptables -I udp2rawDwrW_9bbac27b_C0 -j DROP
[2024-04-28 16:49:08][INFO]run_command iptables -I INPUT -s 203.0.113.100 -p tcp -m tcp --sport 4096 -j udp2rawDwrW_9bbac27b_C0
[2024-04-28 16:49:08][WARN]auto added iptables rules
[2024-04-28 16:49:08][INFO]source_addr is now 192.168.0.110
[2024-04-28 16:49:08][INFO]using port 47991

But in your lastest post you removed.

[2024-06-10 15:39:48][INFO]const_id:cc9eb324
[2024-06-10 15:39:48][WARN] -a has not been set, make sure you have added the needed iptables rules manually
[2024-06-10 15:39:48][INFO]source_addr is now 192.168.1.108

why do you make the change?

I've added the rules myself.

In the at-server-side section it says I need to enable ip forwarding and apply that NAT rule. Is it really needed?

@wangyu-
Copy link
Owner

wangyu- commented Jun 10, 2024

In the at-server-side section it says I need to enable ip forwarding and apply that NAT rule. Is it really needed?

It depends on your use case. Typically:

  1. if you want to redirect 0.0.0.0 you need ipforward
  2. if you wiregaurd ip is private (e.g. 10.0.0.x) and you want to connect to internet via VPN you need NAT

@wangyu-
Copy link
Owner

wangyu- commented Jun 10, 2024

I think i am likely not going to follow up this issue in further, since many newbie mistakes and questions were showing up.

@wangyu-
Copy link
Owner

wangyu- commented Jun 10, 2024

If you still want to make it work. My suggestion:

I suggest you setup both side with iptables first, and make sure it works first. Then move to nftables and compare what is different.

First, as I mentioned, try stick to iptables until you are successful. (After you make sure it works then you can explore how to switch to nft)

Learn from those article:

  1. https://www.procustodibus.com/blog/2020/11/wireguard-point-to-site-config/ ( an article for wireguard without udp2raw . It explains the NAT and ipforward you mentioned. It's nothing related to udp2raw, it's about how to setup a VPN )

  2. https://gist.github.com/hamid-rostami/5ed34fe1948f40685f7035de36be7035 (for the simple case of wireguard+udp2raw without redirect 0.0.0.0/0)

  3. https://www.procustodibus.com/blog/2022/02/wireguard-over-tcp/ (if you need redirect 0.0.0.0/0. look at this longer article)

(try to understand the articles and adapt to your use case)

@o-alquimista
Copy link
Author

Sorry about that. I'm studying the links you provided.

It's working now, with the server running on OpenWrt again — it's easier there. That means that this is not a bug.

One little problem remains: rst==1 warnings in the logs of both client and server. The wiki says this happens when the packets are not being dropped by the kernel, but see this:

Client nftables:

ip saddr PUBLIC_IPV4 tcp sport 4096 counter packets 1080 bytes 1404000 drop

Server nftables:

tcp dport 4096 counter packets 864 bytes 954448 drop

The rules have hits, and I see that the bytes keep increasing during usage — a sign that it's working.

As I don't see any negative impact on the connection — and I cannot use iptables on OpenWrt — I'm ignoring the warnings for now. Do you think this could be a bug? I can file a separate issue for that.

@wangyu-
Copy link
Owner

wangyu- commented Jun 11, 2024

rst==1 warnings means the iptables rule is not working correctly.

Have you tried to use iptables I suggested in the posts above?

Do you think this could be a bug? I can file a separate issue for that.

I think very likely it's not a bug.

you are not using udp2raw generated iptables rules. Instead you are using the ntf tables rule (that you believe it's equivalent, but in reality it might not.)

It's your nft rule is not working as expected. It's not bug on udp2raw side.

@wangyu-
Copy link
Owner

wangyu- commented Jun 11, 2024

As I don't see any negative impact on the connection

If it doesn't have negative impact on your environemnt then it's fine.

But rst==1 definitely means something is not fully working as expected.

— and I cannot use iptables on OpenWrt

I think ntf has iptables compatible layer

have you tried this? https://openwrt.org/packages/pkgdata/iptables-nft

@o-alquimista
Copy link
Author

o-alquimista commented Jun 11, 2024 •
edited
Loading

you are not using udp2raw generated iptables rules. Instead you are using the ntf tables rule (that you believe it's equivalent, but in reality it might not.)

have you tried this? https://openwrt.org/packages/pkgdata/iptables-nft

You're right. I believed they were equivalent, but they're not. Ran a test with the -a option and the iptables compatibility tools installed, and the warnings are gone.

I realized that iptables commands don't change the nftables system. They're not translated into nftables on-the-fly, as I believed they were. In essence, iptables and nftables each have their own separate rules.

Thanks for the help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@wangyu- @o-alquimista