Path matches but getting [Errno 2] No such file or directory: '/root/usr/local/bin/pg_dump'?

[Niqnil]

Bug Report

Please help us help you by filling out any applicable information in this template and removing the rest!

Describe the bug

I have django and postgres running in docker containers. I checked the path in the postgres container and used it in base.py DBBACKUP_CONNECTORS but still get the error.

postgres container shell

bash-5.1# find . -name pg_dump
./usr/local/bin/pg_dump

base.py settings

DBBACKUP_STORAGE = 'django.core.files.storage.FileSystemStorage'
DBBACKUP_STORAGE_OPTIONS = {'location': os.path.join(BASE_DIR, '../backups')}
DBBACKUP_CONNECTORS = {
    'default': {
        'ENGINE': 'django.db.backends.postgresql_psycopg2',
        "NAME": os.environ.get("SQL_DATABASE"),
        "USER": os.environ.get("SQL_USER"),
        "PASSWORD": os.environ.get("SQL_PASSWORD"),
        "HOST": os.environ.get("SQL_HOST"),
        "PORT": os.environ.get("SQL_PORT"),
        'DUMP_CMD': os.path.join(
            os.environ["HOME"],
            'usr',
            'local',
            'bin',
            'pg_dump'
            )
        }
    }

Error with DBBACKUP_CONNECTORS settings.

root@af06c150ccb6:/code# python manage.py dbbackup
Backing Up Database: postgres
CommandConnectorError: Error running:  /root/usr/local/bin/pg_dump  --host=postgres --port=5432 --username=postgres --no-password --clean postgres 
[Errno 2] No such file or directory: '/root/usr/local/bin/pg_dump'
  File "/usr/local/lib/python3.9/site-packages/dbbackup/utils.py", line 118, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 61, in handle
    self._save_new_backup(database)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 74, in _save_new_backup
    outputfile = self.connector.create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 78, in create_dump
    dump = self._create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 38, in _create_dump
    stdout, stderr = self.run_command(cmd, env=self.dump_env)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 21, in run_command
    return super(PgDumpConnector, self).run_command(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 156, in run_command
    raise exceptions.CommandConnectorError(

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 146, in run_command
    process = Popen(cmd, stdin=stdin, stdout=stdout, stderr=stderr, env=full_env)
  File "/usr/local/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: '/root/usr/local/bin/pg_dump'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/code/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/utils.py", line 118, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 61, in handle
    self._save_new_backup(database)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 74, in _save_new_backup
    outputfile = self.connector.create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 78, in create_dump
    dump = self._create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 38, in _create_dump
    stdout, stderr = self.run_command(cmd, env=self.dump_env)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 21, in run_command
    return super(PgDumpConnector, self).run_command(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 156, in run_command
    raise exceptions.CommandConnectorError(
dbbackup.db.exceptions.CommandConnectorError: Error running:  /root/usr/local/bin/pg_dump  --host=postgres --port=5432 --username=postgres --no-password --clean postgres 
[Errno 2] No such file or directory: '/root/usr/local/bin/pg_dump'

Error without DBBACKUP_CONNECTORS settings

root@af06c150ccb6:/code# python manage.py dbbackup
Backing Up Database: postgres
CommandConnectorError: Error running:  pg_dump  --host=postgres --port=5432 --username=postgres --no-password --clean postgres 
[Errno 2] No such file or directory: 'pg_dump'
  File "/usr/local/lib/python3.9/site-packages/dbbackup/utils.py", line 118, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 61, in handle
    self._save_new_backup(database)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 74, in _save_new_backup
    outputfile = self.connector.create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 78, in create_dump
    dump = self._create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 38, in _create_dump
    stdout, stderr = self.run_command(cmd, env=self.dump_env)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 21, in run_command
    return super(PgDumpConnector, self).run_command(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 156, in run_command
    raise exceptions.CommandConnectorError(

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 146, in run_command
    process = Popen(cmd, stdin=stdin, stdout=stdout, stderr=stderr, env=full_env)
  File "/usr/local/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/local/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'pg_dump'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/code/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/usr/local/lib/python3.9/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/usr/local/lib/python3.9/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/utils.py", line 118, in wrapper
    func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 61, in handle
    self._save_new_backup(database)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/management/commands/dbbackup.py", line 74, in _save_new_backup
    outputfile = self.connector.create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 78, in create_dump
    dump = self._create_dump()
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 38, in _create_dump
    stdout, stderr = self.run_command(cmd, env=self.dump_env)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/postgresql.py", line 21, in run_command
    return super(PgDumpConnector, self).run_command(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/dbbackup/db/base.py", line 156, in run_command
    raise exceptions.CommandConnectorError(
dbbackup.db.exceptions.CommandConnectorError: Error running:  pg_dump  --host=postgres --port=5432 --username=postgres --no-password --clean postgres 
[Errno 2] No such file or directory: 'pg_dump

To Reproduce

Dockerfile

FROM python:3.9.6-slim-buster

# Set environment variables
ENV PYTHONDONTWRITEBYTECODE 1
ENV PYTHONUNBUFFERED 1

# Set work directory
WORKDIR /code

# Install system packages required by Wagtail and Django.
RUN apt-get update --yes --quiet && apt-get install --yes --quiet --no-install-recommends \
    build-essential \
    libpq-dev \
    libmariadbclient-dev \
    libjpeg62-turbo-dev \
    zlib1g-dev \
    libwebp-dev \
 && rm -rf /var/lib/apt/lists/*

# Install dependencies
COPY Pipfile Pipfile.lock /code/
RUN pip install pipenv && pipenv install --system --dev

# Copy project
COPY . /code/

docker-compose.yml

version: '3.8'

services:
    web:
        build: .
        command: python3.9 manage.py runserver 0.0.0.0:8000 --settings=IT.settings.dev
        restart: unless-stopped
        volumes:
            - .:/code
        ports:
            - 8000:8000
        env_file:
            - .env/.dev
        depends_on:
            - postgres

    postgres:
        image: postgres:13.4-alpine
        container_name: postgres
        restart: unless-stopped
        ports:
            - 5432:5432
        env_file:
            - .env/.dev_db
        volumes:
            - postgresdata:/var/lib/postgresql/data/

volumes:
    postgresdata:

Expected behavior

python manage.py dbbackup should work as the path matches?

Screenshots or reproduction

If applicable, add screenshots (errors, example of the behavior, etc.) to help explain your problem or post a link to a repository that replicates the issue.

Versions

django-dbbackup: 3.3.0

Django-dbbackup

External tools

• Python: 3.9.6
• Django: 3.2.7
• wagtail: 2.14.1

[diogosilva30]

Also facing the same problem. Any clean solution to this?

[zhaoruibing]

For my case, django and postgres runs in different containers. The django container doesn't have postgres-client installed, which causes this error.

My solution is to install latest postgres-client in django container from postgres repo.
I added the following lines to my django container. Place these after any apt-get purge commands.

RUN apt-get update && apt-get install -y wget gnupg2
RUN echo "deb http://apt.postgresql.org/pub/repos/apt bullseye-pgdg main" > /etc/apt/sources.list.d/pgdg.list
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -
RUN apt-get remove -y wget gnupg2

RUN apt-get update && apt-get install --no-install-recommends -y postgresql-client

Note 1: The reason for using the latest postgres-client from postgresql.org repo (and don't use apt install postgresql-client) is pg_dump checks database version for mismatch. The client must have a higher version than the database. The default postgres-client version in apt is too old (v13) for my database (v14). So I need to get the latest (v15) from the official postgres repo.

Note 2: I used bullseye-pgdg repo, which matches my django container. Modify this to match your container image.

Note 3: I tried to do multi-stage build container to base my django container off postgres image. It requires me to know which files to copy from postgres to django container. I don't know, so I gave up on this method.