How to manage Airflow Dags in Production | Dags versioning & Deployment
#apacheairflow #airflowatjagex #jagex #apacheairflowforbeginners #airflowinproduction
In this video, I'll discussing on how we (at Jagex) manage and operate more than 100+ ETL pipelines in Airflow efficiently.
00:00 Intro
00:27 Issues we faced
02:08 Airflow & DAGs versioning
05:47 Airflow Deployment Process
06:51 Automated CICD deployment
08:53 Airflow Dag versioning proposal AIP-36
09:11 Outro
Learn Something New Below:
https://maxcotec.com/blog/chip-chick-...
https://maxcotec.com/blog/what-is-tra...
👍 If this video was helpful to you, please don't forget to thumbs-up,
⌨️ leave a comment and
🔗 share with your friends.
⭐ Support our work by hitting that subscribe button.
🔔 hit bell icon to stay alert on more upcoming interesting stuff.
DAG versioning proposal
https://cwiki.apache.org/confluence/d...
Watch video How to manage Airflow Dags in Production | Dags versioning & Deployment online, duration hours minute second in high quality that is uploaded to the channel MaxcoTec Learning 14 October 2021. Share the link to the video on social media so that your subscribers and friends will also watch this video. This video clip has been viewed 10,631 times and liked it 165 visitors.
