pod, compose, networking and ports.... #22919

az-z · 2024-06-05T20:32:10Z

az-z
Jun 5, 2024

Hello,
i spent a few very frustrating days getting containers to work together using podman. I'm relatively new to podman.

Task:
I'm working to understand the effects of using "pod", "expose" and "ports" on inter-service communication.

My extremely simplified solution is:

a. create a podman-compose file ( simplified version is below):


version: "3.10"
services:
nginx_ora:
 image: docker.io/library/nginx
 expose: 80
 volumes:
        - db/mo_sql.php:/usr/share/nginx/html/mo_sql.php

restapi:
 image: localhost/curl_nginx
  ports:
    - 8888:8888

b. port 8888 on the host's firewall is open
c. The test :


[az@dell5000 podman-compose]$ podman-compose -f ./podman-compose.yml  --in-pod my_pod up curl_nginx nginx_ora  -d
1866ed387d3f337114821e6da12434d697d78d9e699483fa99b03820f7d51372
432209f98faf50d134b38682c50e7878e820edfacd963e1a34688650af71374c
eb7cc0439b20187f172fdae582fa8a4f3195a239159773b630c0621f6a9d98b3
[az@dell5000 podman-compose]$ 
[az@dell5000 podman-compose]$ podman ps --pod
CONTAINER ID  IMAGE                                               COMMAND               CREATED        STATUS                  PORTS       NAMES                        POD ID        PODNAME
432209f98faf  docker.io/sailtechinc/nginx_instanclient_21:latest  /usr/sbin/nginx -...  6 seconds ago  Up 7 seconds (healthy)              podman-compose_nginx_ora_1   1866ed387d3f  pod_podman-compose
eb7cc0439b20  localhost/curl_nginx:latest                         nginx -g daemon o...  6 seconds ago  Up 6 seconds                        podman-compose_curl_nginx_1  1866ed387d3f  pod_podman-compose

[az@dell5000 podman-compose]$ podman  exec -it podman-compose_curl_nginx_1 sh
# curl localhost/mo_sql.php       
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx/1.25.5</center>
</body>
</html>

[az@dell5000 podman-compose]$ podman  exec -it podman-compose_nginx_ora_1 sh
sh-4.4# curl localhost/mo_sql.php
DADA

The expectation:
since both containers are in the same pod, I expect the mo_sql.php file that resides in the nginx_ora container to be accessible from restapi container.

Note:

a. the "restapi" container is nothing but a standard nginx image with an added "curl" package.
b. the php "script" has the word "DADA" in it. The key here is that the server returns the file.

Questions

what is wrong with my expectations?
when do I use "expose" in a pod?
how do I create a pod in podman-compose call with port mappings?

I appreciate your guidance on this matter.

REF:
https://www.redhat.com/sysadmin/compose-podman-pods

anoduck · 2024-06-06T11:06:17Z

anoduck
Jun 6, 2024

I think this is a good question, and I came here with a very similar question. I think I have the answer to my question, but I do not want to believe it is the case.

@az-z Take a look at the basic networking tutorial OR the paragraph below. I am curious if you arrive at the same conclusion I am afraid of.

The paragraph that scares me is:

As mentioned earlier, slirp4netns is the default network configuration for rootless users. But as of Podman version 4.0, rootless users can also use netavark. The user experience of rootless netavark is very akin to a rootful netavark, except that there is no default network configuration provided. You simply need to create a network, and the one will be created as a bridge network. If you would like to switch from CNI networking to netavark, you must issue the podman system reset --force command. This will delete all of your images, containers, and custom networks.

To my understanding, because podman uses slirp4netns by default, containers cannot have their ports exposed to the outside network as Docker can. Podman can be configured to provide this feature with netavark, but doing so would require the user to effectively wipe the slate clean and start all over. It is doable, but not the most preferred resolution.

2 replies

az-z Jun 6, 2024
Author

After sleeping on my question, life appears to be somewhat better.

@anoduck , thank you for chiming in. In my situation:

[az@dell5000 log]$ podman network inspect podman-compose_default
[
     {
          "name": "podman-compose_default",
          "id": "a9b706a80f2d58c062d58efa687e4245ddaa2d89a33b8c5360f9b0ab812188f3",
          "driver": "bridge",
          "network_interface": "podman1",
          "created": "2024-06-05T12:03:30.808441454-04:00",
          "subnets": [
               {
                    "subnet": "10.89.0.0/24",
                    "gateway": "10.89.0.1"
               }
          ],
          "ipv6_enabled": false,
          "internal": false,
          "dns_enabled": true,
          "labels": {
               "com.docker.compose.project": "podman-compose",
               "io.podman.compose.project": "podman-compose"
          },
          "ipam_options": {
               "driver": "host-local"
          }
     }
]

I found the chapter "Communicate between two rootless containers in a pod" (https://www.redhat.com/sysadmin/container-networking-podman) very confusing. Here is why :

"... Putting them in a pod allows them to communicate directly over localhost:

$ podman run -dt --rm --pod new:mypod alpine_nginx
e2add389417957f2871503aeda82b1d826db3c8e00a5241528b59bf5060d5d94
[bbaude@localhost libpod (master)]$ podman run -it --rm --pod mypod alpine_nginx /bin/sh
/ # curl http://localhost
podman rulez
"

the way I read is that 2 containers can reach each other by referring to localhost. However, the provided article example shows that "podman rulez" response comes from the container's localhost. Not from the localhost of the previously created container.

Matt's ( @mheon ) last comment here makes much more sense to me (than the article's example) : #10815 :

"... but you are correct that host networking removes the advantage of shared networking for all containers in the pod (instead, you get shared networking for all containers on the system with host networking, and the host itself)."

I sense i need to better understand the differences between slirp and netwark ( in rootless mode) to proceed further.

mheon Jun 6, 2024
Maintainer

Basic explanation:

Containers in a Podman pod always share a network stack, be it root or rootless, slirp4netns or pasta or netavark. This includes forwarded ports (if 8080 is forwarded to the pod, every container in the pod can bind to 8080 - though the first to do so will win, as would happen if you tried to run two HTTP servers on the same port on a host). It also includes localhost traffic. All containers in a pod have the same localhost - meaning a container can bind to 8080 on localhost, and every other container in the pod can reach it via localhost:8080.

The bridge network driver (which uses netavark in the backend) is mostly for groups of containers not in a pod. Without bridge, all inter-container communication for rootless containers has to go through pod localhost or forwarded ports on the host itself. If Container A and Container B aren't in a pod, Container A cannot access Container B except by connecting to the host on any port forwarded to B. Bridge networking makes each container have an IP address again, so now A can connect to any services B is hosting even if they are not forwarded to the host. Unfortunately the IPs are inter-container only, so the host can't connect to them (rootless network has a laundry list of limitations, as you might be able to tell).

az-z · 2024-06-06T20:22:26Z

az-z
Jun 6, 2024
Author

Thank you Matt for clarification. In other words - to use bridge and pod is an anti pattern?

…

On Thu, Jun 6, 2024, 16:04 Matt Heon ***@***.***> wrote: Basic explanation: Containers in a Podman pod always share a network stack, be it root or rootless, slirp4netns or pasta or netavark. This includes forwarded ports (if 8080 is forwarded to the pod, every container in the pod can bind to 8080 - though the first to do so will win, as would happen if you tried to run two HTTP servers on the same port on a host). It also includes localhost traffic. All containers in a pod have the same localhost - meaning a container can bind to 8080 on localhost, and every other container in the pod can reach it via localhost:8080. The bridge network driver (which uses netavark in the backend) is mostly for groups of containers not in a pod. Without bridge, all inter-container communication for rootless containers has to go through pod localhost or forwarded ports on the host itself. If Container A and Container B aren't in a pod, Container A cannot access Container B except by connecting to the host on any port forwarded to B. Bridge networking makes each container have an IP address again, so now A can connect to any services B is hosting even if they are not forwarded to the host. Unfortunately the IPs are inter-container only, so the host can't connect to them (rootless network has a laundry list of limitations, as you might be able to tell). — Reply to this email directly, view it on GitHub <#22919 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABGHHXO2T33CFRAUAXBB3Y3ZGC6DTAVCNFSM6AAAAABI3N5ABCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TMOJTGYYTI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

1 reply

mheon Jun 7, 2024
Maintainer

I wouldn't say so. Bridge + pod is to let multiple pods talk to each other. It's not necessary if everything is in one pod.

anoduck · 2024-06-06T20:59:44Z

anoduck
Jun 6, 2024

@az-z It seems we both needed more sleep. Glad your working things out now.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pod, compose, networking and ports.... #22919

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments 3 replies

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

pod, compose, networking and ports.... #22919

az-z Jun 5, 2024

Replies: 3 comments · 3 replies

anoduck Jun 6, 2024

az-z Jun 6, 2024 Author

mheon Jun 6, 2024 Maintainer

az-z Jun 6, 2024 Author

mheon Jun 7, 2024 Maintainer

anoduck Jun 6, 2024

az-z
Jun 5, 2024

Replies: 3 comments 3 replies

anoduck
Jun 6, 2024

az-z Jun 6, 2024
Author

mheon Jun 6, 2024
Maintainer

az-z
Jun 6, 2024
Author

mheon Jun 7, 2024
Maintainer

anoduck
Jun 6, 2024