AMS Background tasks
Below is a brief explanation about Airflow instalation. See the complete documentation here.
Docker Compose v2.14.0 or newer.
Before starting Airflow for the first time, you need to prepare your environment, i.e. create the necessary files, directories and initialize the database.
On Linux, the quick-start needs to know your host user id and needs to have group id set to 0. Otherwise the files created in dags, logs and plugins will be created with root user ownership. You have to make sure to configure them for the docker-compose:
$ echo -e "AIRFLOW_UID=$(id -u)" > .env
You need to run database migrations and create the first user account. To do this, run.
$ docker compose up airflow-init
Now you can start all services:
$ docker compose up
The docker-compose environment we have prepared is a “quick-start” one. It was not designed to be used in production and it has a number of caveats - one of them being that the best way to recover from any problem is to clean it up and restart from scratch.
The best way to do this is to:
-
Run
docker compose down --volumes --remove-orphans
command in the directory you downloaded the docker-compose.yaml file. -
Remove the entire directory where you downloaded the docker-compose.yaml file
rm -rf '<DIRECTORY>'
. -
Run through this guide from the very beginning, starting by re-downloading the docker-compose.yaml file.