Cloud Composer in a Nutshell

Intro

This post is part of a series of posts with notes as I’m studying for Google’s Professional Data Engineer Certification.

This particular post covers Cloud Composer in a nutshell.

Disclaimer

Please read this disclaimer.

Cloud Composer

  • Managed Apache Airflow
  • Create DAGs (Directed Acyclic Graph)
  • Scheduling:
    • Periodic (pull) (specified in the Airflow GUI - e.g. every day at noon)
    • Event-driven (push) (e.g. when new data is uploaded to GCS)