Installing Apache Airflow on Linux or Mac

First things first, Apache Airflow needs Python to be installed. Preferably, 3.12+ We will also use pip / pip3 command to install Airflow.

Initial setup

1. sudo pip3 install virtualenv (install virtualenv so that we can create different environments and Python installations don't clash)
2. mkdir airflow_workspace (make a separate directory for airflow related work)
3. cd airflow_workspace
4. virtualenv airflow_env (creating virtual environment)
5. source airflow_env/bin/activate (activate the virtual environment so that we can use it. Your prompt will change now)
6. pip3 install apache-airflow (using pip or pip3 python installer, install airflow)

Initialize and setup basics (create the first admin user to access UI etc)

1. airflow db init (initialize database - sqlLite in this basic setup will be used, comes packaged with airflow)
2. mkdir dags (create a directory for storing directed acyclic grpahs - the workflows)
3. Add an Admin user to access the workflow UI -- airflow users create --username admin --password your_password --firstname your_first_name --lastname your_last_name --role Admin --email your_email@some.com
4. airflow users list (list users and confirm our admin user got created)

Start airflow scheduler

1. airflow scheduler (Start scheduler in the same terminal)

Start airflow webserver (UI) in the new terminal (after activating the virtual env)

1. cd airflow_workspace
2. source airflow_env/bin/activate
3. airflow webserver