AWXBackup no PostgreSQL user name specified in startup packet

[rexberg]

Please confirm the following

• I agree to follow this project's code of conduct.
• I have checked the current issues for duplicates.
• I understand that the AWX Operator is open source software provided for free and that I might not receive a timely response.

Bug Summary

AWXBackup fails at the "Write pg_dump to backup on PVC" task.

 TASK [Write pg_dump to backup on PVC] ********************************
fatal: [localhost]: FAILED! => {"changed": true, "failed_when_result": true, "rc": 134, "return_code": 134, "stderr": "pg_dump: error: connection to server at \"awx.example.com\" (10.0.11.185), port 5432 failed: FATAL:  no PostgreSQL user name specified in startup packet\
nconnection to server at \"awx.example.com\" (10.0.11.185), port 5432 failed: FATAL:  no PostgreSQL user name specified in startup packet\ndouble free or corruption (out)\nbash: line 18:    27 Aborted                 (core dumped) PGPASSWORD='<redacted>' pg_dump
 --clean --create -h awx.example.com -U awx -d awx -p 5432 -F custom > /backups/tower-openshift-backup-2024-06-07-091254/tower.db\nTerminated\n", "stderr_lines": ["pg_dump: error: connection to server at \"awxdev.db.axis.com\" (10.0.11.185), port 5432 failed: FATAL:
  no PostgreSQL user name specified in startup packet", "connection to server at \"awx.example.com\" (10.0.11.185), port 5432 failed: FATAL:  no PostgreSQL user name specified in startup packet", "double free or corruption (out)", "bash: line 18:    27 Aborted
     (core dumped) PGPASSWORD='<redacted>' pg_dump --clean --create -h awx.example.com -U awx -d awx -p 5432 -F custom > /backups/tower-openshift-backup-2024-06-07-091254/tower.db", "Terminated"], "stdout": "keepalive_pid: 26\nDumping data from database...
\n", "stdout_lines": ["keepalive_pid: 26", "Dumping data from database..."]}

Looking at the backup pod it uses the latest quay.io/sclorg/postgresql-15-c9s image.

$ kubectl -n awx get pods awxbackup-2024-06-07-test3-db-management -o yaml
  - containerID: containerd://4021a29bcc4a34c79904e5874c63af2758150ce5c69d8400ae746984f68b8d5b
    image: quay.io/sclorg/postgresql-15-c9s:latest
    imageID: quay.io/sclorg/postgresql-15-c9s@sha256:60356d43133031c07142122bf00026a16c4d68fa270838d2e62f1b12af5f2cde

Inspecting that image it has openssl 3.2+ installed which according to #15136 is an issue.

$ kubectl -n awx exec -it awxbackup-2024-06-07-test3-db-management -- sh
sh-5.1$ rpm -qa |grep ssl
openssl-libs-3.2.1-1.el9.x86_64
openssl-3.2.1-1.el9.x86_64

AWX Operator version

2.18.0

AWX version

24.5.0

Kubernetes platform

kubernetes

Kubernetes/Platform version

v1.27.12+rke2r1

Modifications

no

Steps to reproduce

Run the AWXBackup manifest for a latest AWX installation using an external PostgreSQL 12.18 instance.

Example:

---
apiVersion: awx.ansible.com/v1beta1
kind: AWXBackup
metadata:
  name: awxbackup-test
  namespace: awx
spec:
  no_log: false
  deployment_name: awx
  backup_pvc: awx-backup-claim
...

Expected results

pg_dump running successfully.

Actual results

It failed with the following error message.

pg_dump: error: connection to server at \"awx.example.com\" (10.0.11.185), port 5432 failed: FATAL:  no PostgreSQL user name specified in startup packet\

Additional information

No response

Operator Logs

No response

[rexberg]

Seems like the version bump to 3.2+ for openssl was introduced in https://quay.io/repository/sclorg/postgresql-15-c9s/manifest/sha256:4844fa1757c82f9d5b585946ec77d8277225997b57a4fb956cc17e1240506742. So changing the version from latest to 20240424 would probably work.

[rexberg]

I can confirm that changing the version to 20240424 works.

Example using the postgres_image_version spec:

---
apiVersion: awx.ansible.com/v1beta1
kind: AWXBackup
metadata:
  name: awxbackup-test
  namespace: awx
spec:
  no_log: false
  deployment_name: awx
  backup_pvc: awx-backup-claim
  postgres_image: "quay.io/sclorg/postgresql-15-c9s"
  postgres_image_version: "20240424"
...

--------------------------- Ansible Task StdOut -------------------------------

TASK [backup : Write pg_dump to backup on PVC] *********************************
task path: /opt/ansible/roles/backup/tasks/postgres.yml:127

-------------------------------------------------------------------------------
...
----- Ansible Task Status Event StdOut (awx.ansible.com/v1beta1, Kind=AWXBackup, awxbackup-test/awx) -----


PLAY RECAP *********************************************************************
localhost                  : ok=7    changed=0    unreachable=0    failed=0    skipped=9    rescued=0    ignored=0

----------