Airflow | Notion

Airflow version is 1.1.0 in Helm chart

Commands

airflow dags list
# OR v1.1 
airflow list_dags
# because we're having 2 containers 1 is git-sync and the other is webserver
# so we have to use --container flag
kubectl exec -it airflow-dev-webserver-7f6cf6c7-nbpjg --container webserver /bin/bash -n dev

airflow dags unpause dag_id
# OR v1.1
airflow unpause dag_id

DAG Synchronizing - GitSync - SideCar

gitSync:
        enabled: true 
        repo: https://github.com/php1301/WeatherForecastDataPipeline.git
        branch: master
        maxFailures: 0
        subPath: "Airflow/dags"
        wait: 60

Should increase number of containers in scheduler and webserver by 1 (gitsync container added)

Untitled

With SSH (private repo for example)

sshKeySecret: airflow-git-private-dags