Before starting Airflow for the first time, You need to prepare your environment, i.e. create the necessary files, directories and initialize the database.
On Linux, the quick-start needs to know your host user id and needs to have group id set to 0. Otherwise the files created in dags, logs and plugins will be created with root user. You have to make sure to configure them for the docker-compose:
mkdir -p ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)" > .env
See Docker Compose environment variables
For other operating systems, you will get warning that AIRFLOW_UID is not set, but you can ignore it. You can also manually create the .env file in the same folder your docker-compose.yaml is placed with this content to get rid of the warning:
AIRFLOW_UID=50000
On all operating systems, you need to run database migrations and create the first user account. To do it, run.
docker-compose up airflow-init
After initialization is complete, you should see a message like below.
airflow-init_1 | Upgrades done airflow-init_1 | Admin user airflow created airflow-init_1 | 2.2.5 start_airflow-init_1 exited with code 0 The account created has the login airflow and the password airflow.
The docker-compose we prepare is a "Quick-start" one. It is not intended to be used in production and it has a number of caveats - one of them being that the best way to recover from any problem is to clean it up and restart from the scratch.
The best way to do it is to:
Run docker-compose down --volumes --remove-orphans command in the directory you downloaded the docker-compose.yaml file
remove the whole directory where you downloaded the docker-compose.yaml file rm -rf ''
re-download the docker-compose.yaml file
re-start following the instructions from the very beginning in this guide
Now you can start all services:
docker-compose up -d
ssh setup:
ssh-keygen -t rsa -b 4096 -f ./id_rsa cp id_rsa ./plugins/devserver_sshprivatekey.pem