Thanks to everyone who contributed to this release: Ash Berlin-Taylor, Brent Bovenzi, Daniel Standish, Elad, Ephraim Anierobi, Jarek Potiuk, Jed Cunningham, Josh Fell, Kamil Breguła, Kanthi, Kaxil Naik, Khalid Mammadov, Malthe Borch, Ping Zhang, Tzu-ping Chung and many others who keep making Airflow better for everyone. SmoothOperator - Operator that does literally nothing except logging a YouTube link to.airflow dags reserialize command to reserialize dags.Support different timeout value for dag file parsing.This isn’t a comprehensive list, but some noteworthy or interesting small features include: More information can be found here: Reuse of decorated DAGs Other small features Here is an def make_list (): # This can also be from an API call, checking a database, - almost anything you like, as long as the # resulting list/dictionary can be stored in the current XCom backend. You can have a task generate the list to iterate over, which is not possible with a for loop. To create a list of tasks, here you can create the same tasks without having to know the exact number of tasks ahead of time. What this means is that you can generate tasks dynamically at runtime. There’s now first-class support for dynamic tasks in Airflow. □ Docker Image: docker pull apache/airflow:2.3.0Īs the changelog is quite large, the following are some notable new features that shipped in this release. Apache Airflow 2.3.0 contains over 700 commits since 2.2.0 and includes 50 new features, 99 improvements, 85 bug fixes, and several doc changes. Create links for BiqTable operators ( #23164).Add exportContext.offload flag to CLOUD_SQL_EXPORT_VALIDATION.google provider - split GkeStartPodOperator execute ( #23518).google provider - BigQueryInsertJobOperator log query ( #23648).Upgrade to support Google Ads v10 ( #22965).This custom version is based on the public version 6.8.0. The delegate_to param remains available only in Gsuite and marketing platform hooks and operators that don't interact with Google Cloud.įor a full list of changes in the apache-airflow-providers-google, see the changelog from version 8.9.0 to 10.1.1 on the apache-airflow-providers-google page.Ĭloud Composer uses a custom version of the apache-airflow-providers-google package, 2022.5.18+composer. Digest:sha256:b10cbc48f6d021c711c72fef7a0e7e8b077a8e58fc3a0dd517b3b71fc9401401. Impersonation can be achieved by utilizing the impersonation_chain parameter instead. Now, you can create tasks dynamically without knowing in. In this version of the provider package, the deprecated delegate_to parameter is removed from all GCP operators, hooks, and triggers, as well as from Firestore and Gsuite transfer operators that interact with GCS. Dynamic Task Mapping is a new feature of Apache Airflow 2.3 that puts your DAGs to a new level.protobuf=4.22.5 is included, this is the first Cloud Composer version with protobuf version 4.x.Google Ads default API changed from version 12 to 13. This version is based on the public version 10.1.1, with additional fixes to some operators and upgrades to many SDK package dependencies (such as protobuf). The apache-airflow-providers-google package in images with Airflow 2.5.1 and 2.4.3 was upgraded to version 2023.6.6+composer. Save money with our transparent approach to pricing Rapid Assessment & Migration Program (RAMP) Migrate from PaaS: Cloud Foundry, OpenshiftĬOVID-19 Solutions for the Healthcare Industry
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |