Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restore tempfile created: 0 B #463

Open
rsommerard opened this issue Oct 5, 2022 · 2 comments
Open

Restore tempfile created: 0 B #463

rsommerard opened this issue Oct 5, 2022 · 2 comments

Comments

@rsommerard
Copy link

rsommerard commented Oct 5, 2022

Bug Report

Running the dbrestore command raise an error which is due to Restore tempfile created: 0 B.

Describe the bug

I use an FTP server as backup storage. Backup is store without issue on the FTP while the restore fail.
If I download the file and run the command by filling the local path via the -I params it works well.

File is not encrypted, not compressed.

The command fail at:

process = Popen(
cmd,
stdin=stdin.open("rb"),
stdout=stdout,
stderr=stderr,
env=full_env,
)

And here is the traceback:

Finding latest backup
Restoring backup for database 'default' and server 'None'
Restoring: default-me-myproject-2022-10-05-141541.psql.bin
Restore tempfile created: 0 B
Are you sure you want to continue? [Y/n] Y
Traceback (most recent call last):
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/dbbackup/db/base.py", line 157, in run_command
    process = Popen(
  File "/usr/lib/python3.8/subprocess.py", line 808, in __init__
    errread, errwrite) = self._get_handles(stdin, stdout, stderr)
  File "/usr/lib/python3.8/subprocess.py", line 1477, in _get_handles
    p2cread = stdin.fileno()
io.UnsupportedOperation: fileno

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/me/.vscode/extensions/ms-python.python-2022.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/me/.vscode/extensions/ms-python.python-2022.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/me/.vscode/extensions/ms-python.python-2022.14.0/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/me/.vscode/extensions/ms-python.python-2022.14.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/me/.vscode/extensions/ms-python.python-2022.14.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/me/.vscode/extensions/ms-python.python-2022.14.0/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/home/me/myproject/manage.py", line 25, in <module>
    execute_from_command_line(sys.argv)
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
    output = self.handle(*args, **options)
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/dbbackup/management/commands/dbrestore.py", line 68, in handle
    self._restore_backup()
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/dbbackup/management/commands/dbrestore.py", line 118, in _restore_backup
    self.connector.restore_dump(input_file)
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/dbbackup/db/base.py", line 105, in restore_dump
    return self._restore_dump(dump)
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/dbbackup/db/postgresql.py", line 124, in _restore_dump
    stdout, stderr = self.run_command(cmd, stdin=dump, env=self.restore_env)
  File "/home/me/myproject/.venv/lib/python3.8/site-packages/dbbackup/db/base.py", line 180, in run_command
    raise exceptions.CommandConnectorError(
dbbackup.db.exceptions.CommandConnectorError: Error running:  pg_restore --dbname=postgresql://postgres:postgres@localhost:30032/postgres --single-transaction --clean 
fileno

To Reproduce

  1. Run a local FTP server docker run -e FTP_USER_NAME=foo -e FTP_USER_PASS=pass -e FTP_USER_HOME=/home/foo -p 21:21 -p 30000-30009:30000-30009 -d stilliard/pure-ftpd
  2. Add the configuration
DBBACKUP_CONNECTORS = {
    "default": {"CONNECTOR": "dbbackup.db.postgresql.PgDumpBinaryConnector"}
}
DBBACKUP_STORAGE = "storages.backends.ftp.FTPStorage"
DBBACKUP_STORAGE_OPTIONS = {"location": "ftp://foo:pass@localhost:21"}
  1. Backup the db: python manage.py dbbackup
  2. Restore: python manage.py dbrestore

Versions

  • django-dbbackup==4.0.2

  • django-storages==1.13.1

  • Django==3.2.7

  • Python: 3.8

  • OS: Ubuntu 22.04.1 LTS x86_64

Misc

The mediarestore command works with the same params (no compression, not encrypted)

@rsommerard rsommerard changed the title Restore issue with STP storage Restore tempfile created: 0 B Oct 6, 2022
@rsommerard
Copy link
Author

rsommerard commented Oct 6, 2022

FYI: for encrypted and compressed file, the same issue appears. A workaround is to add input_file.seek(0) after the input_file assignation in the decrypt block like so in dbrestore.py file: master...rsommerard:django-dbbackup:fix/0b-tempfile

This workaround only works for encrypted files process.

Here a solution when the file is not encrypted or compressed, don't know if it is a good one: https://github.com/rsommerard/django-dbbackup/commit/2d84111514d30ffb0c554ec7abe78e8fa4d68d1a

@tobiasmcnulty
Copy link
Contributor

Confirming we are seeing the issue when both --uncompress and --decrypt are passed to dbrestore; since there is no input_file.seek(0) in between decrypting and decompressing the file, uncompress() doesn't read anything and hence returns an empty file.

Another workaround that doesn't require a code change is to disable compression when encryption is enabled. In our case that resulted in no measurable difference in the backup file size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants