Before Airflow 2.10, there were some limitations on data lineage tracking. LaNeve said that with the new lineage features, Airflow will be able to better capture the dependencies and data flow ...

Key Takeaways Selecting the right data orchestration tool is important due to differences and trade-offs between the variety of options for building distributed data pipelines. Apache Airflow is ...

PayPal recently described how it standardized on Apache Airflow and Apache Gobblin for implementing its next-gen data movement platform. In a recent blog post on the PayPal engineering blog, Jay ...

Learn More The open-source Apache Airflow 2.9 release is out today, providing users of the widely deployed data orchestration platform with a series of advanced capabilities.

Apache Airflow was originally created by Airbnb in 2014 to be a workflow management platform for data engineering. Since becoming a top-level project at the Apache Software Foundation in 2019, it has ...

Since Apache Airflow is an open-source tool, it plays an instrumental role in making data orchestration a reality by taking pain points away.

Airflow’s source code has been available on GitHub since 2015, and it’s been incubating at the Apache Software Foundation since March 2016. Since then, the project has seen an explosion in interest as ...

Data Engineering In 2025: A New Era Of Speed And Efficiency This year, data engineering will have become the backbone of every digital transformation strategy. With the growing complexity of data ...