Lightweight and easy configurable server backup utility.
coverme acts as a backup data aggregator, it grabs data from different sources and uploads it to Amazon Glacier or Amazon S3. More storage options will be probably available. You're always welcome to fork and make a pull request ;)
- Requires Python 2.7+ or Python 3.4+ installed
- Should work on Linux/MacOS/FreeBSD
- Supports MySQL backups via
mysqldump - Supports PostgreSQL backups via
pg_dump - Supports directory archive
- Uploading backups to Amazon S3 as private objects
- Uploading to Amazon Glacier
- Schedule backups via
crontabor run manually
Also for Amazon services configuration you may need to install AWS command line utility awscli.
Install via pip:
pip install coverme
or if only Python 3.x pip is available:
pip3 install coverme
If you're going to use Amazon services for backup, you'll also need to set up credentials via aws configure or manually, see Tips on Amazon services setup.
Just to make sure that installation is correct run:
coverme --help
Please note! Examples below probably do not provide best security practices! Intermediate backups are stored in shared /tmp dir and backups are not encrypted before upload.
-
Define your backup rules in JSON or YAML config, e.g.
backup.yml:--- defaults: # tmpdir - base directory for temporary files tmpdir: /tmp # cleanup - remove archive after upload or not, default: no cleanup: yes # localdir - optional, directory for local backups localdir: backups # format - optional, default: zip format: gztar backups: - type: database url: postgres://[email protected]:5432/test to: [bucket1] name: postgres-{yyyy}-{mm}-{dd}--{HH}-{MM}.sql # tags - optional, no default tags: myapp, postgres, db - type: database url: mysql://[email protected]/test to: [bucket1] name: mysql-{yyyy}-{mm}-{dd}--{HH}-{MM}.sql # tags - optional, no default tags: myapp, mysql, db - type: dir path: /home/myapp/var/www/uploads to: [glacier1] name: myapp-{yyyy}-{mm}-{dd}--{HH}-{MM} # format - optional, default: zip format: gztar # tags - optional, no default tags: myapp, uploads vaults: bucket1: service: s3 region: eu-west-1 # profile - optional, AWS configuration profile # profile: coverme name: coverme-test glacier1: service: glacier region: eu-west-1 account: NNNNNNNNNNNN name: coverme-test ...
-
Perform test backup using config file
backup.yml:coverme backup -c backup.ymlIf some configutaion error happen, fix it first. For example, may you have to configure AWS credentials with
aws configurefirst. -
Add
covermeto cron jobs:crontab -eMake sure to add PATH environment setting, so cron script could found
pg_dump,mysqldumpand other shell commands:SHELL=/bin/sh PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin # run coverme backups every day at 5:30 30 5 * * * /usr/local/bin/coverme backup -c /home/myapp/backup.ymlReal path to
covermemay be different depending on installation, to get proper full path run:which coverme
Docker container is available as rudyryk/coverme.
Here's an example for using in docker-compose.yml:
# ... your services
coverme:
image: rudyryk/coverme
restart: unless-stopped
depends_on:
- postgres
environment:
PGPASSWORD: ${PGPASSWORD}
AWS_ACCESS_KEY_ID: ${AWS_ACCESS_KEY_ID}
AWS_SECRET_ACCESS_KEY: ${AWS_SECRET_ACCESS_KEY}
volumes:
- ./coverme/config.yml:/etc/coverme/config.ymlUse ~/.pgpass file with 600 permissions, see details here:
http://www.postgresql.org/docs/current/static/libpq-pgpass.html
It's easy to rotate backups using name pattern. For example, specify
mysql/dump-{dd}.sql as name for MySQL dump archive and you will get
sequence of files like that:
mysql/dump-01.sql (for 1st day of month)
mysql/dump-02.sql (for 2nd day of month)
...
mysql/dump-31.sql (for 31st day of month)
So you will get monthly backups rotation. Some months have 31 day while others have 30 or 28/29, but that should not be a real issue in most cases.
Get list of available commands:
$ coverme --help
Usage: coverme [OPTIONS] COMMAND [ARGS]...
Command-line interface for coverme.
Options:
--help Show this message and exit.
Commands:
backup
Get help for backup command:
$ coverme backup --help
Usage: coverme backup [OPTIONS]
Options:
-c, --config TEXT Backups configuration file.Specify '-' to read from
STDIN. [default: backup.yml]
--help Show this message and exit.
pip install awscli
aws configure
or manually save credentials to ~/.aws/credentials:
[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY
[another]
; Just a custom profile called 'another', optional section
aws_access_key_id = YOUR_ANOTHER_ACCESS_KEY
aws_secret_access_key = YOUR_ANOTHER_SECRET_KEY
Use Amazon Identity and Access Management (IAM) service to manage users:
https://console.aws.amazon.com/iam/home#users
You may create vaults using wizard:
https://console.aws.amazon.com/glacier/home
In Amazon Identity and Access Management (IAM) panel on user's details page:
Permissions tab -> Inline Policies (click to expand) -> Create User Policy
Choose Policy Generator and then:
- Select "Amazon Glacier" in dropdown
- Check required pesmissions or mark all
- Specify Glacier vault identifier, e.g.
arn:aws:glacier:eu-west-1:NNNNNNNNNNNN:vaults/coverme-test, vault should be created first
In Amazon Identity and Access Management (IAM) panel on user's details page:
Permissions tab -> Inline Policies (click to expand) -> Create User Policy
Choose Policy Generator and then:
- Select "Amazon S3" in dropdown
- Check required pesmissions or mark all
- Specify S3 bucket resources mask, e.g.
arn:aws:s3:::coverme-test/*
Copyright (c) 2016, Alexey Kinëv [email protected]
Licensed under The Apache License Version 2.0, January 2004 (Apache-2.0), see LICENSE file for more details.