No route to IPv6 gateway added when scatic "far" gateway added.

[RooHTaylor]

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/core/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/core/issues?q=is%3Aissue

Describe the bug
To Reproduce

When adding a static IPv6 "far" gateway, no route is added to the routing table to reach said gateway and therefor all upsteam IPv6 traffic fails with No Route To Host.

I have a /64 network on my WAN and my provider uses a static gateway in a different /64 (same /56) as the default upstream gateway, instead of RAs. I assign a static IPv6 address from the WAN /64 to the WAN interface and apply the settings (issue occurs whether I perform this step first or not). I attempt to add the gateway via the GUI page /ui/routing/configuration with the following configuration options:

Interface: WAN
Address Family: IPv6
IP Address: <GATEWAY_IP>
Upstream Gateway: [X]
Far Gateway: [X]
Disable Gateway Monitoring: [ ]
Disable Host Route: [ ]
Monitor IP: <GATEWAY_IP>
Mark Gateway as Down: [ ]
Priority: 255

After saving I "Apply". In can be immediately seen on the that the newly added gateway status is "Down". Attempting to ping any upstream IPv6 address fails with "No route to host". Inspection of the routing table via /ui/diagnostics/interface/routes or via the command line route -6 show <GATEWAY_IP> show that the gateway address is not in the routing table.

Adding the gateway under "Static IPv6 configuration > IPv6 gateway rules" on /interfaces.php?if=wan makes no difference either.

Adding a route to the address manually via the command route -6 add <GATEWAY_IP> -interface vtnet0 makes everything work as expected. I can ping IPv6 upstream, however the route disappears after every reboot.

Expected behavior

As far as I can tell, a "Far gateway" MUST be available on the link. An on-link route should be added for a gateway address specified as a "Far gateway". This matches the behaviour of IPv4 gateways, which do not suffer from this problem. When a "Far" IPv4 gateway is added, an on-link route to the IPv4 address is added to the routing table.

Describe alternatives you considered

I considered having the route manually added via script at every boot, but the behaviour should be automatic.

Relevant log files

There are no logs, other than a failure to add the default route because there is "no route to host". As far as I can tell it doesn't even try to add an on-link route to the address.

Environment

OPNsense 24.1.8-amd64
FreeBSD 13.2-RELEASE-p11

KVM proxmox-ve: 8.2.0 (running kernel: 6.8.8-1-pve)

[fichtner]

Far doesn’t work for IPv6. I think the GUI disabling it for IPv6 got lost during MVC conversion. CC @swhite2

[spin-lock]

Unlike Linux, FreeBSD IPv6 network stack will refuse gateway that does not reside within its configured prefix.
The workaround is to extend (shorten) your prefix length to include the gateway address, in your case use /56 on your WAN.
You can also try to find the gateway's LLA via ndp and use it instead.

[RooHTaylor]

Unlike Linux, FreeBSD IPv6 network stack will refuse gateway that does not reside within its configured prefix.

That's interesting to know, because when I manually add an on-link route to the far gateway it allows me to set it as the default gateway without any complaints or problems at all. I'm no FreeBSD expert, but in practice it seems like it works the same as IPv4 far gateways when executed from the command line.

[fichtner]

Apparently this magically started working somewhere around FreeBSD 13 but it’s been very few instances where this was required. It doesn’t seem to be the proper way to get things up and running.

Cheers,
Franco

[spin-lock]

@RooHTaylor If you mean on-link by doing something like this: 'route -6 add default -iface vtnet0' then yes, its works.
But if we do it by IPv6 address the gateway need to be the same prefix as the interface address, otherwise FreeBSD's ndp implementation can't find the gateway's layer 2 address as these addresses are not considered as neighbors.

@fitchner I just tested it again with wan address prefix /64 and gateway prefix /48 and still no cigars:

root@molybdenum# route -6 add default xxxx:yyyy:zzzz::1 
route: writing to routing socket: Network is unreachable
add net default: gateway xxxx:yyyy:zzzz::1 fib 0: Network is unreachable

Tested it on both 13.2 and 14.1

[RooHTaylor]

@spin-lock Ahh, I see. Perhaps the reason it works for me is because I split it into two commands. First I tell it where the gateway address can be found, then I tell it to use that address as default gateway.

route -6 add xxxx:yyyy:zzzz::1 -interface xyz
route -6 add default xxxx:yyyy:zzzz::1

It makes sense that it wouldn't know how to find the far gateway without being told specifically where it is first. Albiet, adding the interface to the default gateway command should realistically behave the same way, but I've always done it as two.

@fichtner certainly not the best way to do IPv6, but more than one provider that I've come across has things configured this way.