Issues and question about IPv6

[Folaht]

Hey there!

I'm still trying to get 'safe network nodes' running inside a container,
'safe network' being some sort of a decentralized cloud network.
They are supposed to connect to each other and exchange data.
For safety concerns I want to run them all in containers.

I've succeeded setting up a test network in IPv4,
but I prefer IPv6, so I started a second test network.

Here's what I have so far.

sudo podman ps -a

CONTAINER ID  IMAGE                                          COMMAND     CREATED      STATUS          PORTS                                                             NAMES
3c5356f40a5c  localhost/podman-pause:4.0.0-dev-1645184812                4 hours ago  Up 4 hours ago  2a02:a465:21f:1:f7f2:5947:3e84:c7d5:12000-12001->12000-12001/udp  d42aafbb91f8-infra
bd85ccf83c2a  ghcr.io/safenetwork-community/safenode:latest              4 hours ago  Up 4 hours ago  2a02:a465:21f:1:f7f2:5947:3e84:c7d5:12000-12001->12000-12001/udp  root_node
d46f066886a3  ghcr.io/safenetwork-community/safenode:latest              4 hours ago  Up 4 hours ago  2a02:a465:21f:1:f7f2:5947:3e84:c7d5:12000-12001->12000-12001/udp  join_node_1
_ping ifconfig.me_

As one can see, they run.
However when I enter join_node_1
via sudo podman exec -u root -it join_node_1 /bin/bash
and try to ping the outside, which works on any IPv4 node, I get this:

ping ifconfig.me

ping: bad address 'ifconfig.me'

This is the network configuration of container join_node_1

ip addr

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP qlen 1000
    link/ether 1a:d9:e6:45:73:16 brd ff:ff:ff:ff:ff:ff
    inet6 fdf0:3ef:2f04:306a::2/64 scope global
       valid_lft forever preferred_lft forever
    inet6 fe80::18d9:e6ff:fe45:7316/64 scope link
       valid_lft forever preferred_lft forever

I only have one simple network running.

sudo podman network ls

NETWORK ID    NAME        DRIVER
2f259bab93aa  podman      bridge

However, I'm unsure about my IPv6 setup.
Since IPv4 is default, I copy pasted the subnet and gateway part by from copypasting the results of
podman network create --ipv6 and cat /etc/containers/networks/podman1.json | grep subnet|gateway'`

sudo podman network inspect podman

[
     {
          "name": "podman",
          "id": "2f259bab93aaaaa2542ba43ef33eb990d0999ee1b9924b557b7be53c0b7a1bb9",
          "driver": "bridge",
          "network_interface": "podman0",
          "created": "2022-03-16T06:56:37.768948963+01:00",
          "subnets": [
               {
                    "subnet": "fdf0:3ef:2f04:306a::/64",
                    "gateway": "fdf0:3ef:2f04:306a::1"
               }
          ],
          "ipv6_enabled": true,
          "internal": false,
          "dns_enabled": false,
          "ipam_options": {
               "driver": "host-local"
          }
     }
]

Also, I have questions about IPv6.
The standard IPv4 address for podman (with netavark) seems to be 10.88.0.2.

• Why is the IPv4 address somewhat standard, while the IPv6 addresses wildly fluctuate?
• Why does it always seem to start with fd?
• Why are there only the first 4 slots out of 7 used for the subnet? Assuming the /64 is slot 8.
• Why not use only slot 7, like a58::/64 and a58::1?

[mheon]

Please provide the full output of podman info; it will help to know if you're on Netavark or not.

With Netavark, we use IPv6 Unique Local Addresses for internal networks (which start with FC and FD prefixes), and NAT those to the internet, to allow IPv6 with zero external configuration, similar to how IPv4 is supported.

[mheon]

As for why IPv6 addresses fluctuate: we generate a unique prefix for each ULA subnet, and each container gets a random MAC, so addresses will definitely seem random.

[Folaht]

Oh I see now, fc::/fd:: are the equivalent of 192.168.x.x and 10.x.x.x.
I would have never assumed such a large space would be reserved just for unique local addresses.

[Folaht]

There you go.

sudo podman info

host:
  arch: arm64
  buildahVersion: 1.24.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - pids
  - rdma
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: /usr/bin/conmon appartient à conmon 1:2.1.1-1
    path: /usr/bin/conmon
    version: 'conmon version 2.1.1, commit: 65b6eaaf91ed8fd4ef0d5cd9038cff72667cc6be'
  cpus: 4
  distribution:
    distribution: '"manjaro-arm"'
    version: unknown
  eventLogger: journald
  hostname: Rezosur-uq
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.15.48-1-MANJARO-ARM-RPI
  linkmode: dynamic
  logDriver: journald
  memFree: 3465560064
  memTotal: 3976708096
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: /usr/bin/crun appartient à crun 1.4.5-1
    path: /usr/bin/crun
    version: |-
      crun version 1.4.5
      commit: c381048530aa750495cf502ddb7181f2ded5b400
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +CRIU +YAJL
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /etc/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: /usr/bin/slirp4netns appartient à slirp4netns 1.2.0-1
    version: |-
      slirp4netns version 1.2.0
      commit: 656041d45cfca7a4176f6b7eed9e4fe6c11e8383
      libslirp: 4.7.0
      SLIRP_CONFIG_VERSION_MAX: 4
      libseccomp: 2.5.4
  swapFree: 5965058048
  swapTotal: 5965058048
  uptime: 103h 23m 30.61s (Approximately 4.29 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 0
    paused: 0
    running: 0
    stopped: 0
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 2
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.0.0-dev
  Built: 1645184812
  BuiltTime: Fri Feb 18 12:46:52 2022
  GitCommit: ""
  GoVersion: go1.17.6
  OsArch: linux/arm64
  Version: 4.0.0-dev

[mheon]

networkBackend: netavark

You are, indeed, on Netavark, so what you're describing sounds like normal operation.

[Folaht]

Then why can I not ping to the outside?

Is that supposed to work that way or not?

[mheon]

Bad address could indicate DNS issues. Can you ping IPs? Can you look up domain names with dig inside the container?

[Folaht]

I cannot ping IPs, except for ::1, 127.0.0.1, fdf0:3ef:2f04:306a::2 and fdf0:3ef:2f04:306a::1

[Folaht]

Bad address could indicate DNS issues. Can you ping IPs? Can you look up domain names with dig inside the container?

Apparently I can now, but cannot look up domain names with dig.

bash-5.1# ping 2606:4700:3032::6815:4216
PING 2606:4700:3032::6815:4216 (2606:4700:3032::6815:4216): 56 data bytes
64 bytes from 2606:4700:3032::6815:4216: seq=0 ttl=55 time=35.222 ms
64 bytes from 2606:4700:3032::6815:4216: seq=1 ttl=55 time=33.649 ms
64 bytes from 2606:4700:3032::6815:4216: seq=2 ttl=55 time=33.354 ms
64 bytes from 2606:4700:3032::6815:4216: seq=3 ttl=55 time=33.834 ms
^C
--- 2606:4700:3032::6815:4216 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 33.354/34.014/35.222 ms
bash-5.1# dig ufile.io

; <<>> DiG 9.16.29 <<>> ufile.io
;; global options: +cmd
;; connection timed out; no servers could be reached

[Folaht]

Does netavark not support IPv6 without IPv4?
Because I want to use IPv6 exclusively due to the app I run on it uses either IPv6 or IPv4 exclusively
and I prefer IPv6.

sudo podman inspect root_node | grep NetworkSetting -A 48

          "NetworkSettings": {
               "EndpointID": "",
               "Gateway": "",
               "IPAddress": "",
               "IPPrefixLen": 0,
               "IPv6Gateway": "fdf0:3ef:2f04:306a::1",
               "GlobalIPv6Address": "fdf0:3ef:2f04:306a::2",
               "GlobalIPv6PrefixLen": 64,
               "MacAddress": "52:a0:ca:57:97:e4",
               "Bridge": "",
               "SandboxID": "",
               "HairpinMode": false,
               "LinkLocalIPv6Address": "",
               "LinkLocalIPv6PrefixLen": 0,
               "Ports": {
                    "12000/udp": [
                         {
                              "HostIp": "2a02:a465:21f:1:f7f2:5947:3e84:c7d5",
                              "HostPort": "12000"
                         }
                    ],
                    "12001/udp": [
                         {
                              "HostIp": "2a02:a465:21f:1:f7f2:5947:3e84:c7d5",
                              "HostPort": "12001"
                         }
                    ]
               },
               "SandboxKey": "/run/netns/netns-ad134717-3e76-c80f-2d9c-c80021bfd726",
               "Networks": {
                    "podman": {
                         "EndpointID": "",
                         "Gateway": "",
                         "IPAddress": "",
                         "IPPrefixLen": 0,
                         "IPv6Gateway": "fdf0:3ef:2f04:306a::1",
                         "GlobalIPv6Address": "fdf0:3ef:2f04:306a::2",
                         "GlobalIPv6PrefixLen": 64,
                         "MacAddress": "52:a0:ca:57:97:e4",
                         "NetworkID": "podman",
                         "DriverOpts": null,
                         "IPAMConfig": null,
                         "Links": null,
                         "Aliases": [
                              "9660fcd1d460"
                         ]
                    }
               }
          },

[mheon]

Single-stack v6 should be supported. Can you ping the v6 gateway?

[Folaht]

Like this?

bash-5.1# ping fdf0:3ef:2f04:306a::1
PING fdf0:3ef:2f04:306a::1 (fdf0:3ef:2f04:306a::1): 56 data bytes
64 bytes from fdf0:3ef:2f04:306a::1: seq=0 ttl=64 time=0.371 ms
64 bytes from fdf0:3ef:2f04:306a::1: seq=1 ttl=64 time=0.248 ms
64 bytes from fdf0:3ef:2f04:306a::1: seq=2 ttl=64 time=0.236 ms
64 bytes from fdf0:3ef:2f04:306a::1: seq=3 ttl=64 time=0.241 ms
^C
--- fdf0:3ef:2f04:306a::1 ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 0.236/0.274/0.371 ms

[Folaht]

Looks like I'm able to ping IPv6 addresses, just not the domain names.
And because of that, I cannot download the network config file, which I need to get the nodes to connect to each other.