Language:
Undetermined
Pages:
1 online resource (1 video file)
DDC:
006.3/12
Keywords:
Data mining
;
Cloud computing
;
Programming languages (Electronic computers)
;
Exploration de données (Informatique)
;
Infonuagique
;
Instructional films
;
Nonfiction films
;
Internet videos
;
Films de formation
;
Films autres que de fiction
;
Vidéos sur Internet
Abstract:
Apache Airflow Fundamentals Course Overview n this hands-on learning experience, you will gain expertise in building, monitoring, and maintaining data pipelines for workflow orchestration. This course is ideal for data engineers, analysts, and developers who want to programmatically author, schedule, and monitor complex workflows. We'll guide you through Airflow's key concepts, equipping you with real-world skills to automate and orchestrate data pipelines. Get started by installing Airflow using one of the recommended methods like Docker, Kubernetes, or PyPI. We'll ensure you have a working environment to start developing pipelines. Leverage Airflow's user-friendly UI to monitor workflow executions, gain insights, and optimize your pipelines. Master fundamental concepts like directed acyclic graphs, operators, and task dependencies. Learn how to define workflows as code using Python and the rich Airflow API. Develop skills in handling errors, retries, and monitoring pipeline health. Build upon your foundation by scheduling workflows, setting dependencies, and executing production-grade pipelines. We'll explore real-world scenarios like data manipulation and persistence from remote sources. Airflow's plugins and extensibility will allow you to enhance pipelines with custom functionality. Our comprehensive curriculum includes hands-on labs, interactive exercises, and projects focused on real-world data engineering workflows. You'll gain expertise across the entire pipeline development lifecycle - from authoring to monitoring, maintenance, and beyond. Whether you're a beginner or an experienced engineer, this course provides the complete guide to wield the full power of Apache Airflow. Learning Objectives Gain expertise in Airflow concepts like DAGs, tasks, operators, and dependencies to build robust data pipelines. Master Airflow installation methods like Docker and PyPI for development and production environments. Learn to leverage Airflow's UI to monitor workflow executions, analyze failures, and optimize pipelines. Develop skills in authoring workflows as code with Python and utilizing Airflow's extensive APIs. Apply real-world skills through hands-on labs focused on data ingestion, transformation, and ML model deployment. Gain end-to-end experience across pipeline authoring, scheduling, monitoring, alerting, and maintenance. These compelling learning objectives will guide you through the course, helping you acquire essential skills and knowledge to become confident in working with Apache Airflow Course Content This course has an example respository with a lab challenge you can use to apply everything you've learned. The short videos are focused on small examples at a time and contains hands-on demonstrations. The course is designed to be taken in order, but you can jump to any lesson you want to learn more about. Reference GitHub Repository Complete your learning with the practice lab where you will build a data pipeline using Apache Airflow to extract census data, transform it, and load it into a database based on certain conditions. About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to build robust data pipelines with Apache Airflow. Resources Pytest Master Class Practical MLOps book.
Note:
Machine-generated record
Permalink
