Apache Airflow is an open-source platform for developing, scheduling, and monitoring batch-oriented workflows. Airflow's extensible Python framework enables you to build workflows connecting with virtually any technology. A web interface helps manage the state of your workflows. Airflow is deployable in many ways, varying from a single process on your laptop to a distributed setup to support even the biggest workflows. It is a de-facto standard for Data Engineering & ETL pipelines
External link for Apache Airflow
Apache Airflow reposted this
Chief Strategy Officer at Astronomer, Apache Airflow Committer & PMC member, Board member
Airflow 3.0 will be the biggest Apache Airflow release in the last five years. I am thrilled to be part of the large community working on this and excited by the progress we are making.
It is the season to get creative with DAGs! 🎄 Check out this amazing creation by Evie Crutchley - an inspiring blend of art and workflow automation. Who said data pipelines can’t be fun? 🚀✨ #Airflow #ApacheAirflow #DagArt
We've just released Apache Airflow 2.10.4 🎉 📦 PyPI: https://lnkd.in/dK6aFeAA 📚 Docs: https://lnkd.in/d4Awf9pv 🛠 Release Notes: https://lnkd.in/dyY5Nxyf 🐳 Docker Image: "docker pull apache/airflow:2.10.4" Thanks to all the contributors who made this possible.
Apache Airflow reposted this
Happy incoming holidays to all data engineers! Have you put up your holiday decorations yet? 😊 (link to the code in the comments) #airflow #apache #opensource #dataengineering
A friendly reminder that there are only 2 Weeks left to complete the Airflow Survey before it closes on December 20th! Completing the survey takes less than 7 minutes and comes with the following rewards: 💰 A free Airflow Fundamentals Certification or DAG Authoring Certification (valued at $150 each) 🏆 Entry into a raffle for a complimentary workshop with Airflow expert Marc Lamberti Take the survey here! https://lnkd.in/gZHVFj4y #survey #apacheairflowsurvey
Curious about what's coming in Airflow 3? In this episode of the Data Flowcast, PMC Member Vikram Koka chats all things Airflow 3— including the ability to run tasks from anywhere, at any time, in any language. Listen on Apple, Spotify, Youtube, or where you like to 🎧 https://lnkd.in/gqnqVu7N
Join Jarek Potiuk at OSA Con 2024 to discuss all things Airflow 3! https://osacon.io/
🎊 Exciting news— we're offering a FREE Airflow Fundamentals or DAG Authoring Certification to anyone who completes the Airflow Survey! Plus, you'll be entered into a raffle for a free DAG Writing Best Practices Workshop with Marc Lamberti 🙂 Please take the survey here and receive your perks! https://lnkd.in/gZHVFj4y
We've just released Apache Airflow 2.10.3 🎉 📦 PyPI: https://lnkd.in/d8XfEC27 📚 Docs: https://lnkd.in/dkCZYKim 🛠 Release Notes: https://lnkd.in/dJaM36_s 🐳 Docker Image: "docker pull apache/airflow:2.10.3" Thanks to all the contributors who made this possible.
🤔 Have you taken the Annual Airflow Survey yet? 🕑 Don't worry- there's still time to join the 600+ Airflow users who have already taken the time to share their thoughts on the future of the project! 💫 It takes less than 7 minutes, and you will receive a comped Airflow Fundamentals or DAG Authoring Certification ($150 value each), AND be entered into a raffle for a DAG Writing Workshop with Marc Lamberti! Take the survey here and impact the future of Airflow: https://lnkd.in/gZHVFj4y