Abstract: Responsible Generative AI and Local LLMs Learning Generative AI During Live Coding This video series covers mastering the theory of Generative AI and using local LLMs like Mistral, llamafile, Candle, and Lorax. Lessons covered include: Profit Sharing Concepts Tragedy of the Commons Game Theory and Generative AI Perfect Competition Negative Externalities Fine-tuning Mistral with Ludwig Getting Started with llamafile Hello World in Candle Exploring Rust and Candle Transcribing Audio with Whisper Rust for LLMs Learning Objectives Learn about responsible and ethical uses of generative AI Getting hands-on with local LLMs like Mistral, llamafile, Candle, and Lorax Learning to build solutions by coding along during tutorials Learning the key syntax and features of the Rust Language Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: MySQL for Data Engineering Learning Linux Hacker Style Coding Description: Extract MySQL data through queries and bash pipelines Dump query results to CSV files. Build web services to serve data with Python standard library Analyze and process extracts with Linux utilities Chain MySQL, Python, and Linux together for quick data tasks Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Introduction to Generative AI Get started with Generative AI and Large Language Models This introductory course is designed for beginners with no prior knowledge of generative AI. You will start by gaining a high-level understanding of what generative AI is and how it works. Through interactive lessons and hands-on examples, you will learn fundamental skills like providing effective prompts and iteratively improving the generated outputs. As the course progresses, you will dive deeper into specific major generative AI models, including their unique capabilities and limitations. Finally,, you will get practical experience using leading systems like GitHub Copilot, Qdrant, and OpenAI to generate code, and text. By the end, you will have developed core knowledge to start experimenting with generative AI in a responsible and effective way for a variety of applications. This course aims to provide a friendly introduction to prepare complete beginners for further exploration of this rapidly evolving technology. Learning objectives Learn to utilize Generative AI effectively Develop Generative AI software solutions Build solutions with Prompt Engineering to enhance Generative AI output Practice Labs Use the practice labs to test your knowledge and gain hands-on experience working with Python and Rust. These labs are based on GitHub templates that will help you to get started quickly and apply the knowledge in the course in building your own solutions. Learn Retrieval Augmented Generation: https://github.com/alfredodeza/learn-retrieval-augmented-generation Azure Chat with OpenAI: https://github.com/alfredodeza/azure-chat-demo Resources Azure LLMOps Practical MLOps book Rust Bootcamp Python Bootcamp.

Abstract: Interactive Technical Projects A weekly updated series of tutorials on interactive technical projects Learning Objectives Learn practical skills by building real-world projects with cutting-edge technologies Learning to build full stack solutions with languages like Rust, Python, LLMs and Data Science Learning the key tools and techniques used by industry experts to create robust applications Get hands-on practice applying your core technical knowledge Understand how all the pieces fit together by building projects end-to-end Troubleshoot bugs and solve complex problems that appear in production systems Expand your portfolio and r©♭sum©♭ with impressive projects Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: GenAI and LLMs on AWS Learn to build machine learning pipelines leveraging large language models on AWS for natural language processing and generative AI. In this hands-on course, you will gain skills to: * Set up cloud-based Rust development environments * Construct serverless workflows with AWS Lambda and Step Functions * Generate text and code with models like Claude and CodeWhisperer * Orchestrate models and data workflows with Amazon Bedrock * By completing real-world coding projects, you will be prepared to operationalize LLMs for machine learning applications. Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Small Language Models and LLamaFile Learning About Small Language Models and Running Them Locally with LLamaFile This video series covers small language models, their advantages, and how to run them locally using the LLamaFile tool. Lessons Covered Include: 1.1 Introduction to Small Language Models 1.2 Demo: Running Phi Small Language Model with LLamaFile 2.1 Getting Started with LLamaFile 2.2 Demo: Running Lava Multi-Modal Vision Model with LLamaFile Learning Objectives Understand what small language models are and their advantages Learn how to run local language models using the LLamaFile tool See demos of running different small language models like Phi and Lava Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Agile with AI Enhancing Agile Methodologies with Artificial Intelligence This video course explores the integration of Agile practices with Artificial Intelligence (AI) to streamline software development processes and improve productivity. By combining the principles of Agile and the capabilities of AI, you can unlock new levels of efficiency and innovation. Lessons Covered Include: 1.0 Course Introduction 2.1 Waterfall Planning 2.2 Playground vs. Plant Analogy 3.2 Pragmatic Agile 3.15 Productivity Techniques 4.1 AIOps (Artificial Intelligence for IT Operations) 4.2 Diagram Prompt Engineering 4.3 Summarizing Text with Claude 4.4 AI Pair Programming: Paradigm Shift 4.5 AWS CodeWhisperer for Rust 4.7 Serverless Bedrock Diagram 4.8 Bedrock Knowledge Agent: Retrieval-Augmented Generation (RAG) 4.9 Building Blocks for Generative AI: WhisperCPP RAW Learning Objectives Understand the foundations of Agile methodologies and their importance in software development Learn techniques for enhancing Agile practices with AI tools and techniques Explore AI-powered productivity techniques for streamlining development workflows Discover AI-driven approaches to pair programming, code generation, and prompt engineering Gain insights into cutting-edge AI applications in software development, such as AIOps and knowledge agents Learn to leverage AI tools like AWS CodeWhisperer and Retrieval-Augmented Generation (RAG) models Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: This comprehensive course teaches you to leverage the power of cloud computing to develop production-ready machine learning solutions. Starting with the fundamentals of cloud storage, security, and infrastructure as code, you will gain a solid foundation in cloud computing concepts. The course then dives into serverless machine learning, teaching you to build and deploy ML models using popular cloud AutoML platforms like Google AutoML, Amazon SageMaker, Microsoft Azure ML, and CreateML. Through hands-on labs, you'll learn to train custom models with just a few lines of configuration, without needing deep ML expertise. Next, the course explores serverless technologies like AWS Lambda, Google Cloud Functions, and Azure Functions to cost-effectively operationalize your ML models at scale. You'll learn to expose your AutoML models as serverless microservices, automatically scaling to handle any load. The course also covers serverless data processing and ETL to build complete ML pipelines. Finally, you'll learn to apply DevOps best practices to your serverless ML applications using continuous integration and delivery (CI/CD), infrastructure as code, monitoring, and containerization technologies like Docker and Kubernetes. By the end, you'll have the skills to build highly scalable end-to-end machine learning solutions leveraging the latest cloud AutoML and serverless technologies.

Abstract: This video series covers key concepts and technologies for virtualizing and containerizing applications. It introduces the benefits of virtualization, compares virtual machines and containers, and provides a deep dive into working with Docker containers. Through live demonstrations, you'll learn to create and manage virtual machines, pull Docker images, and run containerized applications.

Abstract: Introduction to LLM vulnerabilities This introductory course on vulnerabilities for Large Language Models (LLMs) and language models in general. It provides a deep dive into the practical applications of large language models (LLMs) using Azure's AI services. Upon completion, learners will be able to: Explain the concept of model replication or model shadowing as a potential attack vector in large language models, and describe methods to mitigate it through techniques like rate limiting and buffering. Analyze the potential benefits and limitations of using pre-trained LLMs Develop strategies for mitigating risks and ethical considerations when deploying LLM-powered applications. Describe the high-level process of creating a large language model, including data collection, cleaning, and training. Explain the role of security in large language models and recognize potential security vulnerabilities and attack vectors. Identify insecure plugin designs in large language model software development kits (SDKs) that could lead to remote execution and implement strategies to secure plugins. You will learn how to secure your large language model (LLM) applications by addressing potential vulnerabilities. You will explore strategies to mitigate risks from insecure plugin design, including proper input validation and sanitization. Additionally, you will discover techniques to protect against sensitive information disclosure, such as using a redaction service to remove personally identifiable data from prompts and model responses. Finally, you will learn how to actively monitor your application dependencies for security updates and vulnerabilities, ensuring your system remains secure over time. Week 1: Foundations of Language Models This week you will get a brief overview of LLMs and how do they work Learning Objectives Analyze common types of generative applications and their architectures, including multi-model applications, and understand their challenges and benefits. Explain the functioning of a multi-model application, including the role of the framework and specialized machine learning models. Identify the advantages of smaller, specialized models in terms of resource usage, interaction speed, and deployment agility. Compare and contrast different generative AI application types, such as API-based, embedded models, and multi-model applications, and understand their use cases and challenges. Recognize the importance of large language models in various real-world applications, including text-based chatting, customer service, content creation, and daily tasks. Evaluate the benefits and drawbacks of large language models, considering aspects like accuracy, privacy, and potential misuse. Understand the basics of tokenization, indexing, and probability machines in the context of large language models. Describe the high-level process of creating a large language model, including data collection, cleaning, and training. Explain the role of security in large language models and recognize potential security vulnerabilities and attack vectors. Week 2: Language Model Vulnerabilities This week focuses on model-based vulnerabilities that you can explore with prompts. Learning Objectives Explain the concept of model replication or model shadowing as a potential attack vector in large language models, and describe methods to mitigate it through techniques like rate limiting and buffering. Identify and demonstrate insecure output handling in large language models, and understand the potential security threats and attack vectors associated with it. Understand prompt injection and its implications for large language models, including how certain applications define the initial behavior of these models and how to exploit implicit system prompts. Recognize model theft vulnerabilities and understand how handling and access to system components can impact model security, particularly in the context of dynamically loaded models from external sources. Week 3: System vulnerabilities This week you will learn how to deal with environments and system-based vulnerabilities as they relate to LLMs. Learning Objectives Identify insecure plugin designs in large language model software development kits (SDKs) that could lead to remote execution and implement strategies to secure plugins. Explain the potential risks of sensitive information disclosure in large language models and implement measures to redact personal identifiable information using HTTP APIs and regular expressions. Monitor and update dependencies in large language model applications to prevent potential security vulnerabilities and automate the process using tools like GitHub's Dependabot. Evaluate application vulnerabilities based on the programming language and framework, and implement measures to prevent potential security threats. Week 4: Other types of vulnerabilities Learning Objectives Identify potential security threats and vulnerabilities associated with large and small language models. Implement strategies to prevent security situations and guard against making environments more secure. Recognize the concept of excessive agency in large language models and its potential impacts on functionality. Explain the denial of service threat for large language models and describe methods to guard against API misuse.% About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to understand and work with vulnerabilities within language models. Resources Introduction to Generative AI Responsible Generative AI and Local LLMS Practical MLOps book.

Abstract: Google Professional Cloud Architect Certification Course 2023 (Rough Draft) Welcome to the Google Professional Cloud Architect Certification Course! This course is designed to help you prepare for the Google Cloud Certified Professional Cloud Architect certification exam. As a professional cloud architect, you will be responsible for designing, developing, and managing secure, scalable, and highly available cloud solutions using Google Cloud Platform. Course Overview Course One: Designing and planning a cloud solution architecture 1.0 Rough Draft Introduction Introduction to Google Professional Architect Course Onboarding to GCP Cloud Designing a solution infrastructure that meets business requirements Designing a solution infrastructure that meets technical requirements Designing network, storage, and compute resources Using Google Cloud Functions Running Minikube Running Minikube with FastAPI for Kubernetes development Getting Started with Google Cloud Run Load testing with locust Design technical solutions for evolution and migration SRE Mindset Conclusion and Next Steps Course Two: Managing and provisioning a solution infrastructure Introduction to Google Professional Architect Course Two Exploring Network Topologies in GCP: A Demonstration Embracing Hybrid Networking in Google Cloud Platform Overview of GCP Storage Demo of using the SDK and CLI to manage storage Data lifecycle management Data security Demo of building Rust deduplication finder Compute volatility configuration (preemptible vs. standard) Demo Build and Deploy Rust Actix Microservice Cloud Run Container orchestration with GKE Demo of extending Google Cloud Functions Conclusion Course Two Course Three: Designing for Security and Compliance Introduction to Google Professional Architect Course Three Mastering Identity and Access Management in GCP Data Security Best Practices for GCP Secrets Manager API: A Hands-on Guide Exploring a data poisoning attack Meeting Industry Certification Requirements in GCP Auditing GCP Security with Log Analysis Creating Custom Log Dashboards for GCP Security Monitoring Scanning Web Applications with GCP's Cloud Web Security Scanner Analyzing GCP Logs with BigQuery GCP Security: Conclusion and Next Steps Course Four: Designing and optimizing technical and business processes Introduction to Google Professional Architect Course Four Continuous Delivery: An Introduction Hands-on with Continuous Integration in GCP Troubleshooting Techniques: The Five Whys Implementing Simple GitHub Actions for CI/CD The Three Most Important Files in a Python Project Utilizing Trello for Effective Project Management Leveraging Spreadsheets for Project Management Exploring the MLOps Maturity Model Building Bespoke Systems for Core Business Functions Data Science on Windows: Virtualenv and Pip Site Reliability Engineering (SRE) in GCP Performing Load Testing in GCP: A Demo Course Five: Managing and optimizing cloud solution Implementation Introduction to Google Professional Architect Course Five Makefiles Made Easy: A Comprehensive Guide Configuring Your Bashrc for an Optimal Environment Implementing Continuous Integration for Rust with GitHub Actions Unit Testing in Rust: A Hands-on Demonstration Supercharging Rust Development with Copilot Setting Up a GCP Workstation for Python Development Mastering Google Cloud Shell: A Practical Guide Streamline Your Development with Google Cloud Editor Getting Started with the Google Cloud CLI SDK Unleashing the Power of Google Cloud's 'gcloud' Command-Line Tool Deploying Rust Applications to App Engine: A Step-by Course Six: Ensuring solution and operations reliability Introduction to Google Professional Architect Course Six Monitoring and Logging Rust App Engine Applications: A Demo Uncovering the Advantages of Cloud Developer Workspaces Operationalizing Microservices in the Cloud Introducing GitHub Codespaces for Streamlined Development Compiling Python Projects in GitHub Codespaces Mastering Continuous Integration in GCP Course Wrap-up and Future Steps in Your GCP Journey Learning Objectives Design and optimize cloud solutions Analyze and optimize technical and business processes Manage and provision cloud infrastructure Ensure solution and operations reliability By the end of this course, you will have the knowledge and skills needed to pass the Google Cloud Certified Professional Cloud Architect exam and to effectively design and manage cloud solutions using Google Cloud Platform. Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Introduction to MLflow for MLOps Learn how to use MLflow for managing the machine learning lifecycle. Track experiments, package models, and deploy to production. In this course you'll learn how to use MLflow - an open source platform for managing the machine learning lifecycle. You'll learn how to: Install MLflow and explore its components like the UI, tracking, and model packaging Log metrics, parameters, and artifacts to track ML experiments Create reproducible ML projects with MLflow for repeatable model training Package models and dependencies for deployment and serving Use model registries to version, stage, and deploy models Deploy models to tools like Azure ML and SageMaker This course includes hands-on exercises, projects, and real-world examples so you can apply your new MLflow skills immediately. Use the reference repository for MLFlow examples and projects: Example MLFlow Projects Learning objectives Install and configure MLflow Use the tracking UI and APIs Log metrics, parameters, tags, and artifacts Create reproducible ML projects Version, stage, and deploy models with registries Deploy models to Azure ML, SageMaker, etc Lesson 1: Introduction to MLflow Lesson Outline Overview of MLflow components Installation and configuration Tracking experiments with UI, Python, R APIs Logging metrics, params, tags, artifacts Lesson 2: MLflow Projects Lesson Outline Motivation for reproducible ML projects Creating project directories Running projects locally or on Git Customizing execution environments Lesson 3: MLflow Models Lesson Outline Packaging models and dependencies Model versioning with registries Staging and promoting model stages Deploying models to services About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to work with MLFlow and apply it to MLOps tasks. Resources Pytest Master Class Practical MLOps book.

Abstract: Hands-on Python for MLOps Build powerful APIs, tools, and scripts with Python Use Python to create APIs and automation scripts for machine learning operations. Learn to leverage APIs, build CLI tools, and develop web services. If you are new to Machine Learning Operations or want to know more about the foundations of Python for MLOps, this course will walk you through the basics. In this course, you'll learn how to use Python for automation, APIs, and building useful command-line interfaces. We'll cover consuming APIs, creating CLI tools, packaging projects, and building web services to expose machine learning models. The course includes hands-on examples and projects so you can apply what you learn right away. By the end of the course, you'll have practical skills to automate workflows and develop interfaces for machine learning. This course includes GitHub repositories you can reference: Python CLI Examples Basic Python CLI Web API with Python and Hugging Face models Learn objectives Consume and use APIs and SDKs in Python Create command-line programs for automation Develop web services and APIs with Python frameworks Package Python projects for distribution Apply skills to build useful interfaces for ML models Lesson 1: Working with APIs in Python Lesson Outline Overview of APIs and SDKs Using the Requests Library Consuming REST APIs Python SDKs like NumPy and SciPy Project: Building a Python script using APIs Lesson 2: Building Command-line Interfaces Lesson Outline Intro to automation with CLI tools Parsing arguments and options Organizing code into modules Python packaging for distribution Project: CLI tool for machine learning Lesson 3: Developing Web APIs Lesson Outline REST API concepts Web frameworks like Flask and FastAPI Building an API with Flask Developing APIs with FastAPI OpenAPI specs and documentation Project: Web API for movie recommendations About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his past experience as a Linux system administrator is seen throughout this course, where you will get a first-hand experience with high-level knowledge and practical examples. Resources Pytest Master Class Practical MLOps book.

Abstract: AWS Certified Security - Specialty Exam Prep This video series provides comprehensive preparation for the AWS Certified Security - Specialty (SCS-C02) exam. Lessons Covered Include: Domain 1: Threat Detection and Incident Response 1.0 Introduction 1.1 Designing and Implementing Incident Response Plans 1.2 Detecting Security Threats and Anomalies with AWS Services 1.3 Responding to Compromised Resources and Workloads 1.4 Automating Incident Response with AWS Lambda 1.5 Conducting Root Cause Analysis with Amazon Detective 1.6 Capturing Forensics Data from Compromised Resources 1.7 Querying Logs to Validate Security Events 1.8 Preserving Forensic Artifacts with S3 Object Lock 1.9 Preparing and Recovering Services After Incidents 1.10 Incident Response Case Study 1.11 Practice Exam - Threat Detection and Response 1.12 Threat Detection and Incident Response Summary Learning Objectives Understand best practices for incident response in the cloud Explore AWS services to detect security threats Learn how to respond to compromised resources Domain 2: Security Logging and Monitoring 2.0 Introduction 2.1 Designing and Implementing Monitoring and Alerting 2.2 Troubleshooting Security Monitoring and Alerting 2.3 Designing and Implementing Logging Solutions 2.4 Troubleshooting Logging Solutions 2.5 Designing Log Analysis Solutions 2.6 Configuring AWS CloudTrail for Logging 2.7 Analyzing Logs with Amazon Athena 2.8 Monitoring Case Study 2.9 Practice Exam - Logging and Monitoring 2.10 Optimizing Log Storage Lifecycles 2.11 Third Party Log Analysis Tools 2.12 Security Logging and Monitoring Summary Learning Objectives Understand how to set up monitoring and alerts for security Learn best practices for logging and log analysis Gain skills in troubleshooting logging and monitoring Domain 3: Infrastructure Security 3.0 Introduction 3.1 Designing and Implementing Edge Service Security 3.2 Designing and Implementing Network Security 3.3 Designing and Securing Compute Workloads 3.4 Troubleshooting Network Security 3.5 Securing EC2 Instances with SSM 3.6 Scanning Images for Vulnerabilities 3.7 Infrastructure Security Case Study 3.8 Practice Exam - Infrastructure Security 3.9 Integrating AWS Firewall Manager 3.10 Applying Security Best Practices to Lambdas 3.11 Hardening AMIs with EC2 Image Builder 3.12 Infrastructure Security Summary Learning Objectives Learn how to secure AWS network infrastructure Understand options for compute and workload security Gain skills in troubleshooting infrastructure issues Domain 4: Identity and Access Management 4.0 Introduction 4.1 Designing Authentication for AWS Resources 4.2 Designing Authorization for AWS Resources 4.3 Troubleshooting with IAM Access Analyzer 4.4 Enforcing Least Privilege Access 4.5 Separating Duties with IAM Roles 4.6 Identity and Access Management Case Study 4.7 Practice Exam - Identity and Access Management 4.8 Securing Root User Credentials 4.9 IAM Policy Simulator for Troubleshooting 4.10 Enabling Federated Access with SSO 4.11 Managing Credentials and Secrets 4.12 Identity and Access Management Summary Learning Objectives Master IAM best practices for authentication and authorization Understand how to apply least privilege and separation of duties Learn to troubleshoot IAM issues Domain 5: Data Protection 5.0 Introduction 5.1 Data in Transit Confidentiality and Integrity 5.2 Data at Rest Confidentiality and Integrity 5.3 Managing Data at Rest Lifecycles 5.4 Protecting Credentials, Secrets, and Keys 5.5 Encrypting Data with AWS KMS 5.6 Data Protection Case Study 5.7 Practice Exam - Data Protection 5.8 Enforcing Data Retention with Glacier 5.9 Preventing Data Deletion with S3 Object Lock 5.10 Rotating RDS Credentials 5.11 Encrypting Data in Transit with SSL/TLS 5.12 Data Protection Summary Learning Objectives Learn techniques to protect data in transit and at rest Understand how to manage data lifecycles Gain skills in managing credentials and secrets Domain 6: Management and Security Governance 6.0 Introduction 6.1 Strategies for Central Account Deployment 6.2 Secure and Consistent Cloud Deployment 6.3 Evaluating Compliance of AWS Resources 6.4 Identifying Security Gaps through Reviews and Cost 6.5 AWS Organizations SCP Best Practices 6.6 Governance Case Study 6.7 Practice Exam - Governance 6.8 Automating Security Deployments 6.9 Tagging Resources for Governance 6.10 AWS Config for Security Assessments 6.11 AWS Well Architected Tool 6.12 Governance and Security Management Summary Learning Objectives Understand best practices for governing accounts and resources Learn how to align security with organizational requirements Gain skills in compliance, auditing, and deployment Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification AWS Solutions Architect Professional (SAP-C02) 2023 Assimilate Python From Zero.

Abstract: Rust LLMOps: A Rough Draft Course (Updated JIT) Learning During Live Coding This video series is a rough draft course being updated JIT (Just in Time) and covers the Rust language through live coding. We offer an iterative approach to understand and master the language, particularly focusing on its application in MLOps. Lessons Covered Include: Rust Hugging Face Candle Description This week, you will delve into the powerful combination of Rust with Candle, a minimalist ML framework, and explore how they can be used with Hugging Face's popular transformer models. You will apply these concepts by working on a series of hands-on labs that guide you through building, training, and deploying machine learning models using Rust, Candle, and Hugging Face. Learning Objectives By the end of this lesson, you will be able to: - Implement Candle in Rust to create and train machine learning models. - Utilize the various features of Candle, including syntax, GPU efficiency, and browser deployment. - Integrate Hugging Face transformers with Rust for natural language processing tasks. - Apply the principles of Rust and Candle to build real-world machine learning applications. - Evaluate the performance and scalability of Rust-based ML solutions using Candle and Hugging Face. Lesson Structure Original 4.1 Candle: A Minimalist ML Framework for Rust 4.2 Using GitHub Codespaces for GPU Inference with Rust Candle 4.3 VSCode Remote SSH development AWS Accelerated Compute 4.4 Building Hello World Candle 4.5 Exploring StarCoder: A State-of-the-Art LLM for Code 4.6 Using Whisper with Candle to Transcribe 4.7 Exploring Remote Dev Architectures on AWS.

Abstract: GitHub Foundations: Privacy, Security and Administration (Rough Draft) Updated (JIT) as it is built About This Video Course Building Secure and Efficient GitHub Workflows This video series covers live coding the GitHub Administration and Security domain iteratively, thus learning the best practices and nuances of managing and securing GitHub projects. Lessons Covered Include: 1.0 Course Introduction: Building Secure and Efficient GitHub Workflows 1.1 Introduction to Authentication and Security: Ensuring Your GitHub Account's Integrity 1.2 Securing Account with 2FA 1.3 Understanding Access Permissions 1.4 Enterprise Managed Users (EMUs) Explained 1.5 Introduction to GitHub Administration: Configuring and Managing Repositories and Organizations 1.6 Enabling and Disabling Features 1.7 Repository Permission Levels Explained 1.8 Setting Repository Visibility Options 1.9 Repository Privacy Settings and Options 2.0 Features and Options in the Security Tab 2.1 Understanding Repository Insights 2.2 Managing Collaborators in GitHub 2.3 Managing Organization Settings 2.4 Members, Teams, and Roles in a GitHub Organization 2.5 Next Steps: Continuing Your GitHub Mastery Journey Learning Objectives Learn GitHub's authentication and security measures Learning to build solutions with robust repository management Learning the key syntax and features of GitHub Administration and Security Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification AWS Solutions Architect Professional (SAP-C02) 2023 Assimilate Python From Zero.

Abstract: GitHub Enterprise - Certified Learning the GitHub Enterprise Certification Material This course covers all seven domains of the the GitHub Enterprise Certification Lessons Covered Include: Domain 1: Support GitHub Enterprise for users and key stakeholders Distinguish admin tasks vs needing GitHub Support Generate support bundles and diagnostics Identify underutilized features, integrations, active teams/repos Recommend standards for workflows, branching, code review, automation Explain tooling ecosystem and CI/CD strategy Recommend workflows and tooling to teams Use GitHub APIs to extend admin capabilities Find Marketplace assets for specific needs Contrast GitHub Apps vs actions Domain 2: Manage user identities and GitHub authentication Implications of enabling SAML SSO for orgs/enterprise Steps to enable and enforce SAML SSO Require two-factor authentication Supported identity providers GitHub identity management and authorization Authentication and authorization model Supported SCIM providers How SCIM and team sync works Domain 3: Describe deployment, distribution, licensing Capabilities of GHES, GHEC, GHAE Billing models including licenses, Actions, Packages License usage statistics for orgs, enterprise, machine accounts Consumption of metered products Domain 4: Manage access and permissions Enterprise permissions and policies Organization permissions Team permissions Repository permissions Domain 5: Enable secure development and compliance How GitHub supports enterprise security Scrubbing sensitive data Choosing policies for control required Impacts of policies Organization and enterprise policies Use audit log APIs to explain missing assets Security features of GitHub repositories Create security response plan SSH keys and deploy keys Domain 6: Manage GitHub Actions Distribute actions and workflows in enterprise Manage runners Manage encrypted secrets Domain 7: Manage GitHub Packages Supported Packages Accessing, writing, and sharing Packages Using Packages in workflows Contrasting Packages and releases Learning Objectives Manage user access and permissions across GitHub Enterprise environments through authentication, organizations, teams, and repositories. Establish secure development practices and compliance controls with GitHub features. Distribute and manage GitHub Actions across the enterprise. Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Microsoft Azure AI Fundamentals (AI-900) Certification Pass the AI Fundamentals certification This early release (ongoing) course will give you everything you need to pass the Azure AI Fundamentals (AI-900) certification. This is the foundational certification for doing Machine Learning and Data Science using Azure and its specialized services. The course will go into every content that is relevant to pass the certification including how to create a study guide and strategy to pass the exam. This course includes and covers all of the author's study notes so that you can use them in addition to your own notes . Course overview This course contains a complete walkthrough over all of the content relevant for the AI-900 certification with a summary and specialized notes after each sections, following the same content that is available as Learning Paths. Aside from the video content, create your own study notes and strategy using the example reference guide Course Content Get Started with AI Get Started with AI Study notes Get Started with Artificial Intelligence on Azure About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his experience studying and passing the AI-900 is seen throughout this course, where you will get a first-hand experience with study notes, example repositories, and a solid study strategy. Resources Pytest Master Class Practical MLOps book.

Abstract: This video series covers live coding with TensorFlow and the Rust language.

Abstract: Deploy Rust on Azure Functions Learn how to deploy Rust functions to Azure Functions with this comprehensive video course. You will start by creating a simple HTTP-triggered function, and then use GitHub Actions to automate your deployment. Deploying Rust in Azure can be frustrating because there is no first-class support yet for it. You will learn what are some of the caveats and what are some strategies you can use to properly deploy a very basic Rust HTTP API. Course Overview We will start by covering the basics of Azure Functions and Rust, and then dive into more advanced topics such as configuring your function's environment, application settings, and using external crates. By the end of this course, you will have a working knowledge of how to deploy and manage your Rust functions on Azure Functions using automation to quickly re-deploy or update your function. Course Content Introduction to Azure Functions - Learn the basics of Azure Functions and how they work with Rust. Creating Your First Azure Function in Rust - Create a simple HTTP-triggered function in Rust and deploy it to Azure Functions. Configuring Your Function Environment - Learn how to use environment variables to configure your function's behavior. Use GitHub Actions for automation About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. With Alfredo's guidance, you will gain the knowledge and skills needed to deploy and manage your Rust functions on Azure Functions. Resources Deploying containers to Azure Practical MLOps book Automated Azure Resource Cleanup Azure Fundamentals Certification.

Abstract: Scripting with Python and SQL for Data Engineering Learn Python data structures, web scraping, SQL, and MySQL from the ground up. Master essential skills for collecting, storing, and working with data. In this hands-on course for beginners, you'll learn how to: Store and manipulate data using Python lists, dictionaries, JSON Write reusable scripts to process data Connect Python to databases like SQLite and MySQL Query, import, and export data with SQL Scrape and parse websites using Beautiful Soup and Scrapy Persist scraped data to databases and files You'll use the following example repositories to practice: Mapping data in Python Python Scripting Scrapping Basics Key topics include: Mapping, iterating, and persisting data structures Creating modules, scripts, and workflows in Python SQL essentials - queries, statements, aggregations Setting up connections from Python to SQLite and MySQL Scraping data locally and at scale with spiders Storing scraped data to optimize pipelines You'll build your data wrangling skills through practical examples and hands-on coding exercises in every lesson. By the end of the course, you'll have experience building end-to-end data engineering scripts. Whether you're a beginner looking to learn Python and SQL, or want to develop robust data engineering skills, this course will get you started. Enroll now and start collecting, storing, and working with data using Python and SQL You'll gain hands-on experience building Python scripts and SQL queries for common data engineering tasks. This course is divided in 4 weeks: Week 1 Working with Data in Python By the end of Week 1 you'll be able to: Apply Python data structures like lists, dicts Extract data from sources like CSV, JSON Load and persist data using JSON Lesson 1: Data Structures in Python Lesson Outline Lists, tuples, dictionaries Working with pandas DataFrames Loading data files like CSV into data structures Lesson 2: Reading and Writing Data Lesson Outline Reading and writing CSV files Serializing Python objects with JSON Parsing and dumping JSON data Lesson 3: Persisting and Loading Data in Python Lesson Outline Loading data from files Saving data from Python to disk Loading and saving data to JSON Week 2 Python Scripting and SQL By the end of Week 2 you'll be able to: Write reusable Python scripts Use SQLite to persist data Query SQLite databases with Python Lesson 1: Python Scripting Techniques Lesson Outline Writing modular, reusable Python scripts Exception handling and logging Python virtual environments Lesson 2: Python with SQLite Lesson Outline Creating SQLite databases from Python Writing tables with SQLAlchemy Querying SQLite from Python with SQLAlchemy Week 3 Learning Objectives By the end of Week 3 you'll be able to: Scrape and collect data from websites Build scalable scraping scripts Persist scraped data to files/databases Lesson 1: Web Scraping with Python Lesson Outline HTML parsing and structure Using Beautiful Soup for scraping Storing scraped data in Python Lesson 2: Scalable Web Scraping Lesson Outline Scraping best practices Scaling scraping with multiprocessing Storing scraped data in databases Week 4 Learning Objectives By the end of Week 4 you'll be able to: Connect to MySQL from Python Execute SQL statements and queries Import and export data from MySQL Lesson 1: Python and MySQL Lesson Outline Installing MySQL and configuration Connecting Python to MySQL Executing queries and statements Lesson 2: Running SQL queries from VSCode Use Visual Studio Code to build SQL queries Execute and review SQL queries from Visual Studio Code Lesson 3: Importing and Exporting Data Lesson Outline Loading and exporting CSV data Best practices for moving data into MySQL Automating data imports with Python About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his experience teaching and authoring content about DevOps and MLOps will give you everything you need to get started applying these powerful concepts. Resources Python and Rust CLI Tools Linux For Beginners Hands on Python for MLOps.

Abstract: Rust For DevOps Learn how to use and apply DevOps principles using Rust This is an early release of the course. In this course, you will learn the basics of DevOps using the Rust programming language. DevOps helps you build, test, and deploy code faster while creating a robust foundation for your projects and teams. First, we will talk about what DevOps is. You will learn things like automation, monitoring, and working together as a team. Then, we will see how to use DevOps ideas in Rust code. You will learn about version control, testing, containers, and deploying Rust programs. Although this course explains the foundations of DevOps and using Rust which is ideal for beginners, experienced Rust developers will find solid advice in how to apply DevOps best practices to existing applications. Learning Objectives The core principles and practices of DevOps How to apply DevOps concepts to build, package, and deploy applications with Rust The basics of containerization and how to leverage containers for DevOps with Rust Course Content This course is divided into 4 weeks with 3 lessons that contain about 6 videos each. Each video is about 5 minutes long and contains a hands-on demonstration of the concepts covered in the lesson. The course is designed to be taken in order, but you can jump to any lesson you want to learn more about. Week 1: Introduction to DevOps Principles Introduction to DevOps Principles DevOps Considerations for Applications Basics of Containerization Week 2: Implementing monitoring and logging strategies Introduction to monitoring and logging strategies Tools and Services for monitoring and logging Logging with Rust About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to build robust applications and infrastructure with the Rust programming language and for Rust applications. Resources Rust Bootcamp Deploy Rust on Azure Functions DevOps command-line tools in Python and Rust Switching to Rust from Python First Steps with Rust Learning Path.

Abstract: Linux for Beginners A hands on and practical introduction of Linux This early release (ongoing) course will give you a foundational knowledge to quickly get started with Linux. You will learn practical examples packaged in easy-to-follow videos that give you just enough knowledge to use Linux. There is a strong focus on using the terminal exclusively for this course, which is the best way to ensure a wide variety of Linux environments can be used with almost no difference in concepts. The course will not cover deep Linux internals or advanced usage. It will also not go through specialized scenarios that are meant to pass a Linux certification. Learn objectives Use a terminal to navigate the file system Understand different Linux versions and use readily available versions without installing Linux Use a shell with common features Create and edit files using a terminal text editor Manage system processes and services Install and remove packages About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his past experience as a Linux system administrator is seen throughout this course, where you will get a first-hand experience with high-level knowledge and practical examples. Resources Pytest Master Class Practical MLOps book.

Abstract: DevOps command-line tools in Python and Rust Use both Python and Rust to automate systems engineering tasks In this introductory course, you will learn how to build powerful command-line tools using Python and Rust. You will discover how to set up a development environment, incorporate user input, expand functionality using modules and libraries, and optimize performance for common DevOps and Systems Engineering tasks. Whether you're new to programming or an experienced developer, this course will teach you the fundamentals of building command-line tools using Python and Rust. By the end of the course, you'll have the skills to automate tasks and improve the efficiency of your DevOps and Systems Engineering workflows. This course includes GitHub repositories that you can use as a reference for all the learning content: Python CLI Examples Rust CLI Examples Learn objectives Set up a development environment for building command-line tools. Build basic command-line tools. Expand the functionality of command-line tools using modules and libraries. Create a simple command-line tool using Rust and understand the basic structure of a Rust-based CLI tool. Incorporate user input, such as arguments and options. Learn how to organize code into modules and packages, and work with dependencies and libraries in both Python and Rust. About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his past experience as a Linux system administrator is seen throughout this course, where you will get a first-hand experience with high-level knowledge and practical examples. Resources Pytest Master Class Practical MLOps book.

Abstract: Professional Data Engineer Certification Course Section 1: Designing Data Processing Systems 1.1 Storage technology selection 1.2 Data pipeline design 1.3 Data processing solution design 1.4 Data warehousing & processing migration Section 2: Building & Operationalizing Data Processing Systems 2.1 Storage system implementation 2.2 Pipeline building & operationalization 2.3 Processing infrastructure implementation Section 3: Operationalizing ML Models 3.1 Pre-built ML models as a service 3.2 ML pipeline deployment 3.3 Training & serving infrastructure selection 3.4 ML model measurement, monitoring & troubleshooting Section 4: Ensuring Solution Quality 4.1 Security & compliance design 4.2 Scalability & efficiency assurance 4.3 Reliability & fidelity assurance 4.4 Flexibility & portability assurance Learning Objectives Learn to design, build, and operationalize data processing systems that meet business requirements, system requirements, and industry best practices. Learning to build scalable, efficient, and secure solutions with Google Cloud technologies to manage and analyze data at scale. Learning the key syntax and features of the SQL language for handling, querying, and performing operations on structured data in Google Cloud. Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Getting Started with the Zig Programming Language Learning the Language During Live Coding This video series covers the basics of the Zig programming language through live coding, allowing viewers to learn the language iteratively as it is being used. Lessons Covered 1.0 Getting Started Learning Objectives Learn the fundamentals of the Zig programming language Learn to build solutions with Zig Learn the key syntax and features of Zig Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Hugging Face for MLOps Learn how to leverage Hugging Face and its powerful machine learning capabilities. Build, train, and deploy your own models using the Hugging Face platform and libraries. In this course you'll get hands-on with Hugging Face and learn how to: Access and use pre-trained models from the Hub Fine-tune models with your own data Build machine learning pipelines with Hugging Face Transformers Add and version your own datasets Containerize and deploy Hugging Face models Automate workflows with GitHub Actions By the end, you'll have practical experience building, training, and deploying Hugging Face models, including production deployment to the Azure cloud. Learning objectives Find and use pre-trained models Fine-tune models for custom tasks Build ML pipelines with Hugging Face libraries Create and version datasets Containerize models for production Automate workflows for MLOps Lesson 1: Getting Started with Hugging Face Lesson Outline Overview of Hugging Face Hub Browsing models and datasets Using Hugging Face repositories Managing spaces and access Lesson 2: Applying Hugging Face Models Lesson Outline Downloading models from the Hub Using models with PyTorch/TensorFlow Leveraging tokenizers and pipelines Performing inference with Hub models Lesson 3: Working with Datasets Lesson Outline Browsing datasets on Hugging Face Uploading and managing datasets Versioning datasets with dataset cards Loading datasets in PyTorch/TensorFlow Lesson 4: Model Serving and Deployment Lesson Outline Containerizing Hugging Face models Creating inference APIs with FastAPI Deploying to cloud services like Azure Automating with GitHub Actions About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his experience teaching and authoring content about LOps will give you everything you need to get started applying these powerful concepts. Resources MLOps with Databricks Introduction to MLflow for MLOps Hands-on Python for MLOps.

Abstract: Rust Data Engineering: Course Description Are you a data engineer, software developer, or a tech enthusiast with a basic understanding of Rust, seeking to enhance your skills and dive deep into the realm of data engineering with Rust? Or are you a professional from another programming language background, aiming to explore the efficiency, safety, and concurrency features of Rust for data engineering tasks? If so, this course is designed for you. While a fundamental knowledge of Rust is expected, you should ideally be comfortable with the basics of data structures and algorithms, and have a working understanding of databases and data processing. Familiarity with SQL, the command line, and version control with git is advantageous. This four-week course focuses on leveraging Rust to create efficient, safe, and concurrent data processing systems. The journey begins with a deep dive into Rust's data structures and collections, followed by exploring Rust's safety and security features in the context of data engineering. In the subsequent week, you'll explore libraries and tools specific to data engineering like Diesel, async, Polars, and Apache Arrow, and learn to interface with data processing systems, REST, gRPC protocols, and AWS SDK for cloud-based data operations. The final week focuses on designing and implementing full-fledged data processing systems using Rust. By the end of this course, you will be well-equipped to use Rust for handling large-scale data engineering tasks, solving real-world problems with efficiency and speed. The hands-on labs and projects throughout this course will ensure you gain practical experience, putting your knowledge into action. This course is your gateway to mastering data engineering with Rust, preparing you for the next level in your data engineering journey. Learning Rust During Live Coding This video series covers live coding the Rust language iteratively, thus learning the language. Lessons Covered Include: Section 1: Rust Data Structures: Collections Lesson 1: Getting Started With The Modern Rust Development Ecosystem Meet the instructor & Course Overview: 1.0-meet-instructor-course-overview.mp4 Introduction to the AI Coding Paradigm Shift: 1.1-ai-pair-programming-paradigm-shift.mp4 Introduction to cloud-based development environments: 1.2-GitHub-Codespaces-Ecosystem-with-copilot-chat.mp4 Introduction to GitHub Copilot Ecosystem for Rust: 1.3-copilot-enabled-rust.mp4 Prompt Engineering with GCP BigQuery SQL: 1.4-big-query-prompt-engineering.mp4 Introduction to AWS CodeWhisperer for Rust: 1.5-aws-codewhisperer-for-rust.mp4 Using Google Bard to Enhance Productivity: 1.6-using-bard-to-enhance-productivity.mp4 Continuous Integration with Rust and GitHub Actions: 1.7-continuous-integration-rust-github-actions.mp4 Lesson 2: Rust Sequences and Maps Introducing Rust Sequences and Maps: 1.8-rust-sequences-maps.mp4 Print Rust data structures demo: 1.9-Print-Rust-data-structures-demo.mp4 Vector Fruit Salad demo: 1.10-Vector-Fruit-Salad-demo.mp4 VecDeque Fruit Salad demo: 1.11-VecDeque-Fruit-Salad-demo.mp4 Linkedin List Fruit Salad demo: 1.12-Linkedin-List-Fruit-Salad-demo.mp4 Fruit Salad CLI demo: 1.13-Fruit-Salad-CLI-demo.mp4 HashMap frequency counter demo: 1.14-HashMap-frequency-counter-demo.mp4 HashMap language comparison: 1.15-HashMap-language-comparison.mp4 Lesson 3: Rust Sets, Graphs and Miscellaneous Data Structures Analyzing UFC Fighter Network Using Graph Centrality in Rust: 1.16-ufc-graph-centrality.mp4 Storing Unique Fruits Using HashSet in Rust: 1.17-unique-fruits-with-hashset.mp4 Maintaining Sorted and Unique Fruits Using BTreeSet in Rust: 1.18-sorted-unique-fruits-with-btreeset.mp4 Creating a Fig Priority Fruit Salad Using Binary Heap in Rust: 1.19-fig-priority-fruit-salad-with-binary-heap.mp4 PageRank algorithm for sports data: 1.20-pagerank-sports.mp4 Showing shortest path with dijkstra: 1.21-shortest-path.mp4 Detecting Strongly Connected Components: A Deep Dive into Kosaraju's Algorithm: 1.22-strongly_connected_components_with_kosaraju.mp4 Simple Charting of Data Structures in Rust: 1.23-ascii-graphing.mp4 Section 2: Safety, Security and Concurrency with Rust Lesson 1: Rust Safety and Security Features Multi-Factor Authentication: 2.1_Multi-Factor_Authentication.mp4 Network Segmentation: 2.2_Network_Segmentation.mp4 Least Privilege Access: 2.3_Least_Privilege_Access.mp4 Encryption: 2.4_Encryption.mp4 Mutable fruit salad: 2.5-mutable-fruit-salad.mp4 Customize fruit salad with a CLI: 2.6-customize-csv-fruit-salad.mp4 Data Race example: 2.7-data-race.mp4 Lesson 2: Security Programming with Rust High Availability: 2.10-high-availability.mp4 Understanding the Homophonic Cipher: A Cryptographic Technique: 2.11-homphonic-cipher.mp4 Decoding the Secrets of the Caesar Cipher: 2.12-caesar-cipher.mp4 Building a Caesar Cipher Command Line Interface: 2.13-caesar-cipher-cli.mp4 Creating a Decoder Ring: A Practical Guide: 2.14-decoder-ring.mp4 Detecting Duplicates with SHA-3: A Data Integrity Tool: 2.15-sha3-dupe-detector.mp4 Incident Response: 2.16-incident-response.mp4 Compliance: 2.17-compliance.mp4 Lesson 3: Concurrency with Rust Core Concepts in Concurrency: 2.20-core-concepts-concurrency.mp4 Dining Philosophers: 2.21-dining-philosopher.mp4 Web Crawl Wikipedia with Rayon: 2.22-web-crawl-wikipedia-rayon.mp4 Intelligent Chatbot with Tokio: 2.23-tokio-chatbot.mp4 Multi-threaded deduplication with Rust: 2.24-data-eng-with-rust-dedupe.mp4 Energy Efficiency Python vs Rust: 2.25-energy-efficiency-python-rust.mp4 Concurrency Stress test with a GPU: 2.26-building-cuda-enabled-stress-test-with-rust-pytorch.mp4 Host Efficiency Serverless Optimization problem: 2.27-host-efficiency-optimization-problem.mp4 Section 3: Rust Data Engineering Libraries and Tools Lesson 1: Using Rust to Manage Data, Files and Network Storage Process CSV files in Rust: 2.31-process-csv-rust.mp4 Using Cargo Lambda with Rust: 2.33-cargo-lambda-rust.mp4 List files on AWS EFS with Rust: 2.34-rust-efs-lister.mp4 Use AWS S3 Storage: 2.35-use-s3-storage.mp4 Use AWS S3 Storage from Rust: 2.36-use-rust-for-s3-storage.mp4 Write encrypted data to tables or Parquet files: 2.37-Write-encrypted-data-to-tables-or-Parquet-files.mp4 Lesson 2: DataFrames with Rust, Python and Notebooks What is Colab?: 2.40-what-is-colab.mp4 Using Bard to enhance notebook development: 2.41-using-bard-to-enhance-productivity.mp4 Exploring Life Expectency in a Notebook: 2.42-life-expectancy-eda.mp4 Load a DataFrame with sensitive data: 2.43-Load-a-DataFrame-with-sensitive-information.mp4 Using MLFlow with Databricks Notebooks: 2.44-mlops-mlflow-tracking.mp4 End to End ML with MLFlow and Databricks: 2.45-end-to-end-ml-with-mlflow-and-databricks.mp4 Comparing DataFrame Libraries between Rust and Python: 2.46-comparing-dataframe-libs-rust-python.mp4 Lesson 3: Data Engineering Libraries and Tools with Rust Parquet file writing and reading with Rust: 2.50-parquet-read-write-rust.mp4 Arrow & Parquet in Rust: 2.51-arrow-parquet-rust.mp4 Serverless functions with Rust and AWS Lambda: 2.52-serverless-rust-aws-lambda.mp4 Polars library overview: 2.53-polars-library-overview.mp4 Building RESTful APIs with Rocket: 2.54-building-restful-apis-rocket.mp4 Utilizing Async Rust in Web Development: 2.55-utilizing-async-rust-web-development.mp4 Applying Data Cleaning Techniques with Rust: 2.56-applying-data-cleaning-rust.mp4 Deploying Rust Applications in a Kubernetes Environment: 2.57-deploying-rust-apps-kubernetes.mp4 Section 4: Designing Data Processing Systems in Rust Lesson 1: Getting Started with Rust Data Pipelines (Including ETL) Jack and the Beanstalk Data Pipelines: 4.1-jack-beanstalk-building-data-pipelines.mp4 Open Source Data Engineering - Pros and Cons: 4.2-open-source-de-pro-con.mp4 Core Components of Data Engineering Pipelines: 4.3-core-components-data-engineering-pipelines.mp4 Rust AWS Step Functions Pipeline: 4.4-rust-aws-step-functions.mp4 Rust AWS Lambda Async S3 Size Calculator: 4.5-rust-async-s3-size-calculator-lambda.mp4 What is Distroless: 4.6-what-is-distroless.mp4 Demo Deploying Rust Microservices on GCP: 4.7-demo-build-deploy-rust-microservice-cloud-run.mp4 Lesson 2: Using Rust and Python for LLMs, ONNX, Hugging Face, and PyTorch Pipelines Introduction to Hugging Face Hub: 4.10-intro-hugging-face-hub.mp4 Rust PyTorch Pre-trained Model Ecosystem: 4.11-rust-pytorch-pretrained-models-ecosystem.mp4 Rust GPU Hugging Face Translator: 4.12-rust-gpu-hugging-face-translator.mp4 Rust PyTorch High-Performance Options: 4.13-high-performance-pytorch-rust-demo.mp4 Rust CUDA PyTorch Stress Test: 4.14-building-cuda-enabled-stress-test-with-rust-pytorch.mp4 EFS ONNX Rust Inference with AWS Lambda: 4.15-efs-onnx-lambda-rust-inference-mlops.mp4 Theory behind model fine-tuning: 4.16-intro-fine-tuning-theory.mp4 Doing Fine Tuning: 4.17-doing-fine-tuning.mp4 Lesson 3: Building SQL Solutions with Rust, Generative AI and Cloud Selecting the correct database on GCP: 4.20-gcp-optimize-d...

Abstract: Learning Generative AI with AWS CodeWhisperer During Live Coding This video series covers live coding in various languages iteratively, thus learning the language. Lessons Covered Include 1.0 Getting started with Bash and Rust using AWS CodeWhisperer, VSCode, and AWS Toolkit for VSCode Learning Objectives Learn the fundamentals of Generative AI Learn to build AI-powered solutions with Bash, Rust, and AWS services Master the key syntax and features of Bash and Rust languages Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Rust for Pythonistas What you'll learn-and how you can apply it A multi-hour jam session where you will learn to code in Rust. * Write efficient and safe Rust code * Leverage Rust libraries and tools in your projects * Combine Rust with Python for improved application performance Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Rust AWS Lambda Build and deploy serverless applications with Rust and AWS Lambda This course will help you build and deploy serverless applications with Rust and AWS Lambda. You will learn how to create, deploy, and manage AWS Lambda functions using Rust, leveraging the power and performance of the language for serverless architecture, and apply it by building a real-world Lambda function as a part of a distributed application. What you will learn Understand the concept of serverless architecture and AWS Lambda. Create and deploy AWS Lambda functions written in Rust. Integrate Lambda functions with other AWS services. Monitor, test, and optimize Rust-based Lambda functions. Implement real-world applications using Rust and AWS Lambda. Lesson 1: Getting Started with Rust AWS Lambda and Serverless Introduction to Serverless and AWS Lambda Walk through Rust Firecracker Project Setting up VSCode AWS Toolkit and CodeWhisperer for Rust Introduction to Cargo Lambda Rust Cost Advantage AWS Lambda Using the AWS Lambda Console Using the Step Functions Console Invoking Step Functions from AWS CLI Lesson 2: Advanced Techniques with Rust and AWS Lambda Building a Rust AWS Lambda Add Function Building a Rust AWS Lambda Divide by Two Function Invoking AWS Step Function from CLI Building Chainable AWS Step Functions with Rust Serverless MLOPs with EFS mounted in AWS Lambda Using Rust AWS Lambda Function URLs Saving money with arm64 Rust AWS Lambdas Lesson 3: Building a Polars Rust AWS Lambda Solution Polars Rust AWS Lambda Build and Deploy Polars Rust AWS Lambda Using AWS Console with deployed Polars Rust AWS Lambda Analyzing Rust AWS Lambda code with CodeWhisperer and AWS Toolkit AWS Lambda Function URLs Build and Deploy Polars Rust AWS Lambda Function URLs.

Abstract: Assimilate ONNX Learning about ML Model portability During Live Coding This video series covers live coding the Language iteratively thus learning the language. Lessons Covered Include: 1.0 Getting started with ONNX. In this live coding session I was able to get both resnet and ONNX working in Rust with Sonos/Trac. Learning Objectives Learn to work with ONNX models Learning to build solutions with ONNX models in Rust Learning the key syntax and features ONNX models Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: This introductory lesson will walk you through everything you need to know to quickly get started with Azure ML Studio using AutoML. After creating the necessary resources, you'll walk through the steps needed to create a regression job using a dataset. Next you'll create a compute cluster and review the different options available and finally, you'll analyze the results after the best model is selected.

Abstract: Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Certification Exam Guide Welcome to the Google Professional Machine Learning Engineer Course! This course is designed to help you prepare for the Google Professional Machine Learning Engineer certification exam. Learning Objectives Develop a deep understanding of Google Cloud technologies and various ML models and techniques to design, build, and productionize machine learning solutions that address specific business challenges while adhering to responsible AI practices. Collaborate effectively with cross-functional teams, including application developers, data engineers, and data governance professionals, to ensure the long-term success of ML models throughout their development, deployment, and maintenance. Master the skills required to design, implement, and manage ML architectures, data pipelines, and metric interpretations, as well as optimize model performance through training, retraining, deploying, scheduling, monitoring, and refining models in scalable and efficient ways. Learn how to use the Google Cloud Platform (GCP) to build and deploy ML models, including how to use GCP services such as BigQuery, Cloud Storage, Cloud AI Platform, and Cloud Functions to build and deploy ML models. Who Should Take This Course? Data scientists Data engineers Machine learning engineers Software engineers Data analysts Data architects Business analysts Anyone interested in learning about machine learning and Google Cloud Platform Course One: Framing ML Problems Course Two: Architecting ML solutions Course Three: Designing data preparation and processing systems Course Four: Developing ML models Course Five: Automating and orchestrating ML pipelines Course Six: Monitoring, optimizing, and maintaining ML solutions Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Rust Bootcamp Course Overview Welcome to our comprehensive Rust programming course! In this immersive learning experience, we will guide you through every aspect of Rust, empowering you to become a proficient developer. This course is ideal for beginners wanting to become proficient in Rust or from existing developers coming from languages like Python or JavaScript that want to learn Rust fundamentals. Get started by setting up your preferred text editor and installing Rust. We'll ensure that you have all the necessary tools to create a productive coding environment. By leveraging the capabilities of Visual Studio Code and enabling the Rust Analyzer, you'll unlock a feature-rich ecosystem that enhances your development workflow. Experience the game-changing potential of GitHub Copilot, an AI-powered assistant. Sign up for GitHub Copilot and witness its transformative impact on your coding experience. We'll guide you through the installation process and show you how to leverage its intelligent suggestions for accelerated programming. Master concepts and fundamentals like control flow, variable assignment, and immutability. Learn the intricacies of loops, conditional statements, and effective error handling techniques. With Rust's borrowing concept at your disposal, you'll develop secure and high-performance code. Build upon your foundation with advanced topics like structs, string manipulation, and working with vectors and enums. We'll explore real-world library development using Cargo, Rust's robust package manager. Effective documentation and debugging techniques will become second nature as you optimize your code. Organize your code effectively with modules, extending functionality and enhancing reusability. Dive into comprehensive testing methodologies to ensure the reliability and correctness of your programs. From writing tests for your code to handling private components, you'll gain expertise in developing robust software projects. Our immersive learning experience includes hands-on examples, interactive exercises, and practical projects. Rust's performance, safety, and expressiveness will be at your fingertips as you embrace its immense potential. Join us on this rewarding journey to unlock the power of Rust programming and propel your career forward. Enroll now and gain the skills, knowledge, and expertise to thrive in the world of Rust. Whether you're a beginner, coding novice, or an experienced developer seeking new horizons, our course is your gateway to writing elegant, reliable, and high-performance code. Embrace the possibilities of Rust and embark on a rewarding programming journey today with one of the most loved programming languages. Learning Objectives Gain a solid understanding of Rust's core concepts, including variable assignment, control flow, and immutability, to write efficient and reliable code. Master the utilization of powerful development tools such as Visual Studio Code and the Rust Analyzer, enhancing productivity and enabling seamless coding experiences. Explore GitHub Copilot and learn how to leverage its AI-driven suggestions to accelerate programming and boost code quality. Develop proficiency in working with advanced Rust features, such as structs, enums, and string manipulation, to build robust and flexible applications. Discover the intricacies of module organization and extend functionality through effective code structuring, promoting code reusability and maintainability. Acquire comprehensive testing skills, including writing tests for code components and handling private code, ensuring the reliability and correctness of Rust programs. Gain hands-on experience in real-world library development using Cargo, Rust's package manager, and become proficient in documenting code and debugging techniques. These compelling learning objectives will guide you through the course, helping you acquire essential skills and knowledge to become a confident Rust developer. Course Content This course is divided into 4 weeks with 3 lessons that contain about 6 videos each. Each video is about 5 minutes long and contains a hands-on demonstration of the concepts covered in the lesson. The course is designed to be taken in order, but you can jump to any lesson you want to learn more about. Week 1: Setting up your development environment Reference GitHub Repository Setting up your text editor Using GitHub Copilot with Rust Introduction to GitHub Codespaces for Rust Week 2: Rust fundamentals Reference GitHub Repository Introduction to Rust Loops and Control flow Function basics Week 3: Structs, Types, and Enums Reference GitHub Repository Using structured data Exploring strings and vectors Working with Enum and Variants Week 4: Applying Rust Reference GitHub Repository Building a real-world library Extending functionality with modules Testing Rust code About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to build with the Rust programming language. Resources Deploy Rust on Azure Functions DevOps command-line tools in Python and Rust Switching to Rust from Python First Steps with Rust Learning Path.

Abstract: Live Coding in Rust: Unleash the Power of Systems Programming Delve deep into the Rust ecosystem with this groundbreaking live coding series, focused on building robust and performant software, including cutting-edge topics like LLMOps and machine learning with Hugging Face's Candle library. What You'll Get Live coding sessions with expert commentary Weekly updates and continually expanding! High-impact projects and examples Exclusive discussions on Rust best practices Topics covering LLMOps, machine learning, and more A focus on performance, safety, and systems programming Course Outline (Future and Current Content) Rust Fundamentals Introduction to Rust Understanding Ownership & Borrowing Error Handling in Rust LLMOps in Rust Live Coding Rust: LLMOps in Rust--Exploring Hugging Face Candle Building Custom LLMOps Libraries Rust and LLVM: A Match Made in Heaven Machine Learning in Rust Implementing ML Algorithms in Rust Using Hugging Face Candle for NLP Deploying Rust ML Models in Production Systems Programming Writing File Systems in Rust Creating Web Servers with Actix Asynchronous Programming with async/await Rust in DevOps Automating CI/CD Pipelines with Rust Rust for Cloud Operations Rust in Containerized Environments Topics Covered LLMOps : The intersection of LLVM and DevOps Machine Learning : Building and deploying models in Rust Systems Programming : Low-level operations made easier and safer DevOps : Leverage Rust for automating operations tasks Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Using Rust with Python A Practical Guide to Rust for Python Developers Topics: Lesson 1: Introduction to Rust and Python Integration & PyO3 Basics Video 1.1 : Introduction to Rust and Python Video 1.2 : PyO3 Installation Video 1.3 : Basic Rust Library Video 1.4 : Rust to Python Video 1.5 : Rust Ownership Model Video 1.6 : Diagram PyO3 Project Video 1.7 : Python Calc CLI Video 1.8 : PyO3 Features Video 1.9 : PyO3 Exceptions Lesson 2: Advanced Rust and Python Mixing Techniques Video 1.10 : Call Python from Rust Video 1.11 : Run Python Embedded in Rust Video 1.12 : Embedded Rust CLI Diagram Video 1.13 : Embedded Rust CLI Video 1.14 : Embedded Rust CLI Test Video 1.15 : Rust-built Python Tools Video 1.16 : Using Rust Ruff Linter Lesson 3: Python and Rust Command-Line Tools & Recap Video 1.17 : Using Polars with Python and Rust Video 1.18 : Polars CLI in Rust Video 1.19 : Polars CLI Test in Rust Video 1.20 : Polars-CLI Integration Test Video 1.21 : Polars Criterion Benchmarking Learning Objectives Understand the need for Rust and Python integration. Explore the PyO3 library and its usage. Understand the Rust ownership model and its usage in PyO3. Who Should Take This Course Python developers who want to learn Rust and its integration with Python. Rust developers who want to learn how to integrate Rust with Python. Developers who want to learn how to build command-line tools using Rust and Python.

Abstract: Achieving Scalability with Vector, Graph, and Key/Value Databases Description This week we explore vector and graph databases, powerful tools for managing and extracting insights from large, complex datasets. As data volumes continue to grow, scalability is crucial. We'll learn how vector and graph databases can efficiently store data while maintaining relationships, enabling more advanced analytics. Through real-world examples, you'll see how these databases unlock scalability for machine learning, fraud detection, social networks, and more. Learning Objectives Learn Vector, Graph, SQLand Key/Value Databases Learning to build solutions with Rust, Vector, Graph, and Key/Value Databases Learning advanced data engineering Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Live Coding Rust Candle from Hugging Face Learning LLMOps This video series covers live coding Rust Candle from Hugging Face for LLMOps Lessons Covered Include: 1.0 Building binaries for Phi using AWS CodeBuild and Rust Candle Learning Objectives Run your own Rust LLMs Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Databricks Certified Data Engineer Associate Course 1: Databricks Lakehouse Platform Description Learn foundational Databricks capabilities including compute, storage, notebooks, and jobs to build scalable data solutions. Learning Objectives Create clusters and configure runtime environments Perform exploratory analysis with notebooks Schedule and monitor multi-task workflows Course 2: Databricks SQL Description Master Spark SQL for reading, transforming, and loading data at scale. Learn techniques like data validation, custom business logic, and slowly changing dimensions. Learning Objectives Query data in notebooks with Spark SQL Handle complex data types Apply data quality rules Implement slowly changing dimensions Course 3: Databricks ML Description Build ML models with Python and Scala APIs in Databricks. Learn best practices for feature engineering, hyperparameter tuning, and model evaluation. Learning Objectives Engineer features from raw data Tune models with cross validation Evaluate model performance Operationalize models with MLflow Course 4: Databricks Data Engineering Description Architect reliable and performant data infrastructure with Delta Lake, streaming, and autoscaling. Learning Objectives Implement ACID transactions Build streaming ETL solutions Autoscale infrastructure to meet SLAs Migrate data warehouses to lakehouse Course 5: Workloads with Jobs Description Orchestrate workloads using multi-task Jobs with configurable scheduling, dependencies, and error handling. Learning Objectives Schedule notebooks, jobs and pipelines Set dependencies across tasks Handle and retry failures Monitor runs using the Jobs UI Course 6: Data Access with Unity Catalog Description Provide governed data access across storage like ADLS, S3, and GCS using Unity Catalog. Learning Objectives Deploy a Unity Catalog Manage credentials securely Apply object-level security Query data from storage tiers Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: India, the Lifestyle for Environment (LiFE) initiative aims to encourage the adoption of sustainable lifestyles in India and internationally to tackle the challenges of environmental degradation and climate change. This report examines how India has integrated several policies in its energy transition strategy that are aligned with the LiFE initiative, highlighting the potential for behavioural change and consumption choices to help advance energy transitions globally. It then analyses the impact on energy consumption, costs and emissions of measures like those proposed by the LiFE initiative, such as buying an EV or taking public transport, if they were adopted globally. Finally, it considers how India’s first G20 Presidency could strengthen the LiFE initiative by anchoring it in the G20’s current framing of energy transitions and initiating processes to gather experience and best practices of policies and programmes that G20 members are already conducting.

Abstract: This dataset includes energy statistics on the basis of main economic indicators such as Consumer Price Index, Purchasing Power Parities, Producer Price Index and currency exchange rates.

Abstract: This dataset provides wholesale and retail energy end-use price indices by sector for both household and industry in nominal and real terms. Products such as coal, heavy fuel oil, automotive diesel fuel, natural gas, oil products, unleaded gasoline are the core focus of this dataset.

Abstract: Finland has set one of the most ambitious climate targets in the world, a legal obligation to reach carbon neutrality by 2035. It has made notable progress towards this target, deploying the first new nuclear reactor in Europe in over 15 years and strongly expanding wind generation. Thanks to the progress Finland has made on its clean energy transition, the country has the second lowest share of fossil fuels in its energy supply among IEA members. It is also reducing its reliance on Russian energy imports and ensuring energy security by increasing imports from other countries, raising domestic renewable energy production and improving energy efficiency. Despite these notable successes in clean energy and energy security, significant challenges remain. Imported fossil fuels still account for over a third of the energy supply while some areas of the Finnish economy, such as transport and key industrial activities, remain dependant on fossil fuels. Also, land use change and forestry in Finland, which have historically offset a significant amount of greenhouse gas emissions, became a net source of emissions for the first time in 2021. In this report, the IEA provides a range of energy policy recommendations to help Finland smoothly manage the transition to a secure, efficient and flexible carbon neutral energy system.

Abstract: This report was commissioned by Korea's Ministry of Trade, Industry and Energy and carried out jointly by the International Energy Agency (IEA) and the Korea Energy Economics Institute. The objective of the study was to provide high-level policy recommendations on Korea’s clean energy transition from coal in the power sector. The report covers a detailed review of policy and market developments around Korea’s transition to net zero. The scope of the report includes all sectors of the economy, industry and all the regions across the world, where we extract the main recommendations that are applicable to the case of Korea. Currently, the power sector is the largest CO2-emitting sector and coal is the single biggest source of CO2 emissions, as it is the backbone of many electricity systems. Thus, coal power plants have been a target for reaching net zero emissions by 2050 for long time. Korea has firm objectives to achieve carbon neutrality by 2050, aiming to accelerate the clean energy transition of coal power plants. Policy recommendations were formulated around two priorities: affordable and secure supply of electricity and people-centred transition.

Abstract: The Clean Energy Transitions Programme 2022 (CETP) provides an overview of the programme’s major achievements in the last year. During this time, the CETP has driven forward an ambitious and impactful agenda, even as the world has contended with multiple overlapping crises. As just some examples, the CETP delivered the Energy Sector Roadmap to Net Zero Emissions in Indonesia , which ministers strongly welcomed at the G20, and has played a key role in shaping Indonesia’s landmark Just Energy Transition Partnership (JETP). Similarly, the IEA has been able to take its engagement to a whole new level with various African countries, such as with its first-ever report on the Clean Energy Transitions in the Greater Horn of Africa , and provided policy advice that was directly accepted into China’s latest five-year plan. More broadly, it has also supported strategically important work on critical minerals, energy employment and clean energy investment. Overall, the CETP’s combination of data, insight, advice and capacity building is making a strong positive contribution to advancing clean energy transitions around the world – with a focus on emerging and developing economies. In the five years since its launch, the CETP has become the largest and most important programme at theInternational Energy Agency (IEA). It has been the main vehicle through which theIEA has established, deepened and strengthened partnerships with major nonmember countries – those countries that are at the forefront of the clean energytransition. The programme has also played a crucial role in enabling the IEA tostrengthen its global leadership role in clean energy transitions at a pace and scalethat would otherwise not have been possible, with great benefit to theprogramme’s focus countries and IEA member countries alike.

Abstract: The International Energy Agency (IEA) and the Institute of Energy Economic, Japan (IEEJ) have developed and published long-term decarbonisation pathways for Southeast Asia and Indonesia. This report provides a comparison of modelling approaches, quantitative drivers, and results from the IEA and IEEJ pathways, highlighting areas of agreement, as well as identifying and explaining differences, and thereby to derive implications.

Abstract: Towards Hydrogen Definitions Based on their Emissions Intensity is a new report by the International Energy Agency, designed to inform policy makers, hydrogen producers, investors and the research community in advance of the G7 Climate, Energy and Environmental Ministerial meeting in April 2023. This report assesses the greenhouse gas emissions intensity of the different hydrogen production routes and reviews ways to use the emissions intensity of hydrogen production in the development of regulation and certification schemes. An internationally agreed emissions accounting framework is a way to move away from the use of terminologies based on colours or other terms that have proved impractical for the contracts that underpin investment. The adoption of such a framework can bring much-needed transparency, as well as facilitating interoperability and limiting market fragmentation, thus becoming a useful enabler of investments for the development of international hydrogen supply chains.

Abstract: Greece aims to reduce total greenhouse gas emissions by 55% by 2030 and reach net zero emissions by 2050. It has made notable progress towards meeting these targets. This includes significantly reducing its reliance on lignite and setting a binding target to end lignite-fired power generation by 2028 while working to ensure a just transition in its lignite mining regions and reducing energy poverty. Greece has also made strong progress on renewable energy, which covered 20% of its total final energy consumption in 2021. Following Russia’s invasion of Ukraine, Greece is reducing its reliance on Russian energy imports and ensuring secure access to energy by diversifying its supply and increasing domestic energy production. Despite these notable successes, significant challenges remain. Fossil fuels still account for most of Greece’s energy use and stronger efforts are needed on energy efficiency. n this report, the IEA provides a range of energy policy recommendations to help Greece smoothly manage its transition to a secure, efficient and flexible carbon neutral energy system.

Abstract: Countries around the world are facing an energy security and affordability crisis affecting households, businesses and public budgets. In Latin America, household expenditure on energy services is second only to food. On average, the share of household budget for energy ranges between 7% and 9% of income – and up to more than 24% for the poorest households. As governments urgently implement measures to protect citizens and secure their economies, it is important to consider short- and long- term solutions to ease existing pain and reduce risks over time. While much attention focuses on securing energy supplies, it is equally important to take a hard look at the role of more efficient and flexible demand in securing affordable, reliable energy.Decades of policies and programmes have delivered energy savings around the world. As a result, energy systems are more resilient, bills are lower, and public budgets feel less pressure than they would have without these efforts. This report looks at energy efficiency and demand management policies and programmes across Latin America. It presents evidence of the results of these efforts on improving living standards, public budgets, energy security and employment, and provides a view on the opportunities to expand these benefits and support just energy transitions.

Abstract: This dataset provides statistics on electricity imports by country of origin.

Abstract: This table presents detailed coal export data by destination for OECD member states for the following coal products and aggregates: anthracite, coking coal, sub-bituminous coal, lignite/brown coal, coke oven, coke, hard coal, brown coal, steam coal, other bituminous coal and anthracite. The data are reported in kilotonnes.

Abstract: This dataset provides statistics on electricity supply per OECD country with net electricity production from main sources and trade data from January 2000 onwards. Data are presented on a monthly basis.

Abstract: Since the publication of its latest Power Development Plan (PDP) in 2020 (PDP 2018 Revision 1), Thailand has considerably increased its emissions reductions objectives, announcing a net zero greenhouse gas emissions target for 2065 and carbon neutrality for 2050. As the power sector is a large part of the country's emissions, and because it has a key role to play in decarbonising other sectors, meeting these targets is possible only if the power sector is decarbonising too. This report hence analyses how Thailand can achieve its clean electricity transition, by comparing the planned trajectory of the PDP with the emissions targets, and providing an assessment of the gaps. Building upon the current PDP, this report analyses how the Thai power system can decrease its emissions to meet the targets by increasing the amount of wind and solar PV in its system, and how it can integrate these variable renewable energy sources efficiently. This report concludes work area one of the joint work programme among the Electricity Generating Authority of Thailand (EGAT), the Ministry of Energy of Thailand and the International Energy Agency (IEA), and has benefited from data and input from the Thai counterparts. The analysis is based on a PLEXOS model of the Thai power system that has been developed by the IEA in cooperation with EGAT.

Abstract: This dataset includes statistics on energy technology Research & Development and dissemination as well as R&D budget for IEA countries. It presents shifts in R&D, expenditures associated with investments and further analyzes budget allocations in terms of flow.

Abstract: This table presents detailed coal import data by origin for OECD member states for the following coal products and aggregates: anthracite, coking coal, subbituminous coal, ignite/brown coal, coke, oven coke, hard coal, brown coal, steam coal, other bituminous coal and anthracite. The data are reported in kilotonnes.

Abstract: This table contains data on the energy trade, supply and consumption of coal, oil, gas, electricity, heat, combustible renewables and waste, expressed in thousand tonnes of oil equivalent (ktoe).

Abstract: This dataset includes statistics on research & development within the energy technology industry as well as R&D dissemination indicators (exchange rate, PPP,Nominal GDP, GDP deflator). Data coverage commences in 1974.

Abstract: This comparative tables presents figures on the renewable energy supply and consumption market amongst OECD member countries by analyzing renewable products in terms of production, import and exports.

Abstract: This dataset figures reprensenting various renewable and waste energy supply statistics while showcasing the renewable energy market's products as well as figures for those products in terms of electricity output (GWh, heat output(TJ), domestic supply and stock changes.

Abstract: 31 countries (30 individual countries + European Union), 8 products and 11 flows (8 summary groups of energy technologies + Total + Memos: Low-carbon and Non-low-carbon)

Abstract: This table focuses on the import of various energy products such as crude oil, refinery gas, natural gas liquids, fuel oil amongst OECD member countries.

Abstract: This report assesses the impact of the road transport sector on energy demand, CO2 emissions and air pollution in several selected major emerging economies over the coming decades under several IEA modelling scenarios. Most notably the Announced Pledges Scenario (APS) aims to show to what extent announced ambitions and targets, including the most recent ones, are on the path to deliver emissions reductions required to achieve net zero emissions by 2050. Bringing about a road transport decarbonisation pathway in line with the APS in the selected major emerging economies - Brazil, People's Republic of China, India, Indonesia, Mexico and South Africa - will require significant enhancement of existing policies and the introduction of new innovative policies and measures in each of selected countries. Our report sets out six policy areas critical to the achievement of the road transport transitions and a series of recommendations for strengthening financing for the sector. Importantly, the report provides detailed reference to a wide range of policy measures and good practice already in place in many major emerging economies elsewhere to facilitate knowledge sharing among countries. It also places a special emphasis on the road transport sectors of India and Indonesia. These countries are IEA partners in their respective regions and benefit from an enhanced programme of work.

Abstract: Although Africa accounts for one-fifth of the global population, the region currently attracts only 3% of global energy investment. By 2030, energy investment needs to double to over USD 200 billion per year, in order for African countries to achieve all their energy-related development goals, including universal access to modern energy, while meeting in time and in full their nationally determined contributions. Financing Clean Energy in Africa, a World Energy Outlook Special Report, builds on the key findings from the Africa Energy Outlook 2022, which introduced the Sustainable Africa Scenario (SAS), and charts innovative investment solutions across the continent that are critical to scale up energy investment. It develops a theory of change based on the positive spillover effects of increasing the availability of affordable capital for clean energy projects. Currently, the cost of capital for energy projects in African countries is at least 2-3x higher than in advanced economies and China, which hinders investment by raising project costs. The International Energy Agency (IEA) and the African Development Bank Group have joined forces to produce this new analysis, which benefitted from the review of over 85 case studies and over 40 stakeholder interviews. The report focuses on a range of topics, spanning technologies and financing providers, including local institutions, and looks at what types of capital are most suited for the specificities of each sector or technology. The analysis pays close attention to how to scale up private investment, including the role of de-risking support from development finance institutions (DFIs) and donors: by 2030, USD 28 billion of concessional capital will be necessary to mobilise the required USD 90 billion in private investment in clean energy. Increasing the role of the private sector allows DFIs and donors to also scale up support to non-commercial areas, such as enabling environments, unproven technologies and fragile and conflict-afflicted states, unproven technologies.

Abstract: The International Energy Agency (IEA) regularly conducts in-depth peer reviews of the energy policies of its member, accession and association countries. This process supports energy policy development and encourages the exchange of international best practices and experiences. This first energy policy review of Colombia's energy policies examines the country's achievements in developing its energy sector as well as the challenges it faces in ensuring a sustainable energy future. Colombia's energy transition policy making is an inspiring example of a fossil fuel producing country committed to climate action, based on a long-term decarbonisation pathway and a policy of energy and economic diversification and a just transition. This report provides insights into Colombia's unique energy system transformation, which is linked to expanding access to electricity and clean cooking for its citizens and swiftly deploying renewable energy technologies. It analyses the full breadth of the country's energy sector and presents recommendations for strengthening the country's people-centred, secure and clean energy transition. These include clean energy technology and innovation, adapting energy market rules, notably in power and gas markets; integrating higher shares of variable renewables; addressing air quality; and reducing vulnerability to the impacts of climate change.

Abstract: A massive scaling up of investment is essential in emerging and developing economies to sustainably meet rising demand for energy, as well as to ensure that climate targets are met. Getting on track for net zero emissions by 2050 will require clean energy spending in emerging and developing economies to more than triple by 2030 – far beyond the capacity of public financing alone and therefore demanding an unprecedented mobilization of private capital. This special report by the International Energy Agency (IEA) and International Finance Corporation (IFC) examines how to scale up private finance for clean energy transitions by quantifying the investments required in different regions and sectors to build modern, clean energy systems, including achieving universal access. The new global energy economy represents a huge opportunity for growth and employment in emerging and developing economies. This report's analysis identifies key barriers and how to remove them – and sets out the policy actions and financial instruments that can deliver a major acceleration in private capital flows for the energy transition.

Abstract: This table shows a comprehensive status of net electrical capacity by type of energy. It also contains data on single fuel-fired and multi-fuel fired plants, type of generation and peak load.

Abstract: This dataset provides conversion factors in terms of trade, production and consumption of energy within various industries.

Abstract: This table contains data on crude supply by comparing imports, exports, stock changes and trade flows of products such as crude oil, natural gas liquid, biofuels, hydrocarbons and refinery feedstocks.

Abstract: This dataset focuses on the renewables market while providing full balance data such as primary energy supply, transformation sector and final consumption based in kilotonne and terajoule of renewables equivalent for OECD countries and OECD country aggregate.

Abstract: This table includes statistical projections on the energy balance for IEA countries by analyzing their respective products (coal, oil, nuclear, hydro, biofuels and more) in terms of flow. Data are expressed in thousand tonnes of oil equivalent(ktoe) and forecasts extend to 2030.

Abstract: This dataset includes data on savings due to the 3-factor analysis for energy consumption and 4-factor analysis for emissions.

Abstract: As the global energy transition accelerates, all energy sectors will experience significant transformations, affecting how energy is produced and consumed. This implies a changing role of hydrocarbon fuels such as oil and natural gas. While these fuels will continue to play a role in energy systems, notably in large energy demand centres, all scenarios point to a gradual decline in demand for both oil and gas. As a result, producer economies such as Oman, whose economic development has been to a significant part enabled by a thriving oil and gas industry, have been exploring opportunities to adapt their economic model to the emerging energy economy we are observing. Nevertheless, it remains important to appreciate the role producer economies play in the global energy transition. While this transformation involves a wide range of challenges to secure future revenues and employment opportunities for citizens, the energy transition also holds with it a range of opportunities. Oman's assets, infrastructure, and skills were once developed in the service of an omnipresent oil and gas industry. Today, the objective of this study is to lay out the opportunities of repurposing these advantages in the services in developing clean energy supply chains in Oman, led by solar PV, wind and low-emission hydrogen where significant economic value is expected to lie as the global economy decarbonises.

Abstract: Carbon capture, utilisation and storage (CCUS) is an important technology for achieving global net zero emissions. Momentum on CCUS has increased in recent years, but the deployment of projects has remained relatively flat. Emerging business models are opening the door to new investment opportunities, and with that bringing new challenges to be overcome. The scale-up needed to reach net zero emissions by mid-century represents a major undertaking, and policy support and co-ordination are crucial. Policy makers have a suite of tools at their disposal to create the conditions necessary to drive long-term investment, enabling industry to take the next step forward and push CCUS into a viable and sustainable commercial market. This IEA CCUS Handbook provides governments with a policy toolkit to tackle the overarching challenges to CCUS deployment. It gives an overview of existing policies that have helped launch CCUS projects to date and identifies the main challenges to future large-scale deployment. The handbook also highlights international best practices, drawing on existing and proposed government efforts to address these challenges. The handbook is supported by our CCUS Projects Database1 and complements the IEA CCUS Handbooks on Legal and Regulatory Frameworks for CCUS and on CO2 Storage Resources and their Development.

Abstract: Energy efficiency continues to play a critical role in improving living standards around the world and is the first and best response to simultaneously meet affordability, supply security and climate goals. As Kenya looks to drive forward its clean energy transition in the face of the global climate and energy crises, there is a growing role for energy efficiency in supporting its aims to ensure affordable, reliable access to electricity while allowing greater integration of renewable energy technologies. As part of the Energy Efficiency in Emerging Economies (E4) Programme, this report aims to provide an overview of current progress in energy efficiency and its potential for improving people's lives through delivery of a sustainable, modern energy system. The report assesses progress, opportunities and challenges for energy efficiency across four key areas: Buildings, Appliances, Clean Cooking and Electricity System Losses. The report gives suggestions on potential policy actions that can be taken to enhance progress, drawing on case studies and examples from Kenya and other countries in Africa and globally. It represents part of the IEA’s growing collaboration with Kenya in the build-up to the IEA’s Energy Efficiency in Emerging Economies Training Week and 9th Annual Global Conference on Energy Efficiency, which will take place in Nairobi in March and May 2024 respectively.

Abstract: The Lifestyle for Environment (LiFE) initiative, launched in 2022 by India, aims to promote environmentally responsible consumption choices and behaviour in India and worldwide. High level principles on implementing LiFE were adopted by the G20, under India’s presidency. Following the first IEA report on LiFE, which examined the impact on energy consumption, costs and emissions of measures like those proposed by the LiFE initiative, this report provides a concrete policy toolkit for advancing LiFE implementation at the level of countries, subnational jurisdictions and individuals, and through international fora such as the G20. Enabling people to adopt sustainable consumption choices and habits requires dedicated and sustained policy interventions. This report examines the mechanisms of policies in driving change. Of the almost 1.7 billion tonnes (Gt) of CO2 savings that would be achieved in G20 countries by an implementation of LiFE-aligned measures, around 60% could be directly influenced or mandated by policies. It showcases that the LiFE movement is gaining traction, tracking around 120 LiFE-aligned policies already implemented across G20 countries and building policy toolkit based on experiences and best practices. Finally, it examines the multiple co-benefits of LiFE measures on equity, air pollution, employment and economic development, health, water, food and land-use.

Abstract: Today, oil and gas operations account for around 15% of total energy-related emissions globally, the equivalent of 5.1 billion tonnes of greenhouse gas emissions. In the International Energy Agency’s Net Zero Emissions by 2050 Scenario, the emissions intensity of these activities falls by 50% by the end of the decade. Combined with the reductions in oil and gas consumption in this scenario, this results in a 60% reduction in emissions from oil and gas operations to 2030. Fortunately, oil and gas producers have a clear opportunity to address the problem of emissions from their activities through a series of ready-to-implement and cost-effective measures. These include tackling methane emissions, eliminating all non-emergency flaring, electrifying upstream facilities with low-emissions electricity, equipping oil and gas processes with carbon capture, utilisation and storage technologies, and expanding the use of hydrogen from low-emissions electrolysis in refineries. Upfront investments totalling USD 600 billion would be required to halve the emissions intensity of oil and gas operations globally by 2030. This is only a fraction of the record windfall income that oil and gas producers accrued in 2022 – a year of soaring energy prices amid a global energy crisis. This report aims to inform discussions on these issues in the run-up to the COP28 Climate Change Conference in Dubai in November and is part of a broader World Energy Outlook special report to be released later in 2023 focusing on the role of the oil and gas industry in net zero transitions.

Abstract: New Zealand has set ambitious targets for reducing greenhouse gas emissions, including achieving net zero emissions by 2050. The country enjoys many natural advantages for its energy transition, including an enviable renewable resource base. New Zealand already has a low‑emissions electricity system, with significant production from both hydropower and geothermal, and therefore has an attractive opportunity to leverage this clean electricity to decarbonise end-user sectors. This will require not only sizeable technological investments to efficiently electrify transport and industry, but will also necessitate a sizeable buildout of additional renewables generation capacity, along with supplemental grid and storage investments. Notably, the transport sector accounts for the highest share of emissions and is almost entirely dependent on oil. Industry is also a major contributor to New Zealand’s emissions and is heavily reliant on fossil fuels. In this report, the IEA provides energy policy recommendations to help New Zealand effectively manage the transformation of its energy sector in line with its climate targets.

Abstract: Energy efficiency indicators are key to tracking energy efficiency progress for a variety of purposes (e.g. policy making, monitoring targets, making energy projections, developing scenarios and planning, and benchmarking). This guide is for professionals and decision makers, describing options and good practices for the collection of energy end-use data and the development of energy efficiency indicators at the national level. In parallel, it can also be used as an assessment tool, helping countries/economies to locate their starting point, and to identify appropriate targets according to their respective national interests and priorities.The roadmap presented here encompasses the results of a consultation exercise across countries and presents good practices and practical tips. It acknowledges that there is no single solution, but a number of possible pathways instead, depending on national contexts and priorities. The roadmap is a strategic document looking at the whole value chain in the development of efficiency indicators, from the initial point where the need for data and indicators arises up to the later dissemination and data use stages, and is meant to be a useful resource for practitioners across the globe in the development of energy efficiency indicators.

Abstract: This table focuses on the export of various energy products such as crude oil, refinery gas, natural gas liquids, fuel oil amongst OECD member countries.

Abstract: This table presents gross electricity and heat production from each energy source such as nuclear, hydro, combustible fuels, geothermal, solar, wind and other sources as well as the breakdown according to the type of plant (electricity only, CHP or heat only). Inputs by type of fuel are also provided. Gross heat production is the total heat produced by the installation and includes the heat used by the installation's auxiliaries which use a hot fluid (space heating, liquid fuel heating, etc.) and losses in the installation/network heat exchanges whereas Net Heat Production is the heat supplied to the distribution system as determined from measurements of the outgoing and return flows.

Abstract: A future with net zero emissions requires scaling up improvements in energy efficiency, electrification of end uses and renewable energy generation. For the Association of Southeast Asian Nations (ASEAN), a growing population and rising standards of living will massively increase future energy demand. Mitigating growing electricity demand and integrating renewable energy into electricity generation will therefore be paramount for the region's clean energy transition and avoid lock-in of additional fossil fuel generation. The intermittent nature of variable renewable energy and increasing deployment of distributed energy resources are putting additional pressure on existing grids. In response to these challenges, this report explores the opportunities and challenges for efficient grid-interactive buildings in the ASEAN region. Such buildings are becoming a crucial element for the global ambition to attain net zero emissions, as they can combine enhanced energy efficiency, advanced smart digital technologies and decarbonised electricity generation. This integration creates the potential for buildings to shift from energy-intensive consumers to low-carbon prosumers, empowered by digital technologies that can offer flexibility benefits to the electricity grids. With the ability to produce, consume, store, sell and buy energy, buildings become active participants in the building-to-grid ecosystem. Drawing on relevant international trends and best practices, the current report lays out an analytical framework to assess a variety of factors that can enable a building to become energy efficient and grid-interactive. The framework is used to analyse the current situation in countries of the ASEAN region. The report provides policy-oriented recommendations and guidelines tailored to different stages of the process for adopting efficient grid-interactive solutions in buildings. These recommendations can support ASEAN policy makers in their policy development to create an energy-efficient and grid-interactive built environment, contributing to a cleaner and more sustainable energy future.

Abstract: This dataset comprises statistics regarding various renewable and waste energy while presenting trade statistics of various renewable energy products as well as figures for those products in terms of electricity output (GWh) and heat output(TJ).

Abstract: This table contains data on natural gas exports by origin for OECD countries.

Abstract: This table contains world data on supply of natural gas.

Abstract: This table includes monthly OECD net electricity production from main sources and trade data from January 2000 to December 2010.

Abstract: This dataset covers 30 countries and 1 indicator.

Abstract: This dataset provides data on energy world indicators in terms of world supply, consumption and trade.

Abstract: This dataset comprises a detailed set of statistics on carbon dioxide (CO2) emissions estimates from fossil fuel combustion. It includes an extensive set of products emitting CO2 from combustion analyzed in terms of their respective industry flows (iron and steel, manufacturing industries, agriculture, machinery, etc). Data are presented from 1960 onwards.

Abstract: This dataset presents detailed statistics on carbon dioxide (CO2) emissions from fuel combustion by product (coal/peat, oil, gas) and flow (manufacturing, main activity producer, construction and more) according to the 2006 IPCC Guidelines. Data are presented from 1960 onwards.

Abstract: This dataset includes data on Services energy end use, Services activity, Services emissions as well as Services energy and carbon indicators.

Abstract: This dataset includes data on Transport energy end use, Transport activity and Transport emissions as well as Transport energy, carbon and activity indicators.

Abstract: Mauritania has high-quality wind and solar resources whose large-scale development could have catalytic effects in supporting the country to deliver universal electricity access to its citizens and achieve its vision for sustainable economic development. Renewables deployment would benefit mining – the largest industry in the country – which is currently reliant on diesel and heavy fuel oil for its operations. A switch to renewable energy in the sector could lower costs, reduce emissions, increase efficiency and improve energy security in the country. There is also potential to further electrify energy uses in mining. The government has announced various export-oriented projects to produce renewable hydrogen, ammonia and/or hydrogen-reduced iron. Anchoring demand on foreign offtakers would contribute significantly to de-risk these projects and generate the stable revenue stream needed to mobilise investors at the necessary scale. By attracting significant amounts of capital, such large-scale projects could enable a transformation of the power sector and spur sustainable economic development and growth, but robust and transparent policies and regulatory frameworks are needed. This new IEA report – the first focusing on Mauritania – explores the potential benefits to Mauritania of developing its renewable energy options and includes an analysis of the water requirements of hydrogen and the potential for expanding potable water availability through seawater desalination.

Abstract: This in-depth review of the energy policies of Uganda follows the format used by the International Energy Agency (IEA) for its peer reviews for member countries. This process supports energy policy development and encourages the exchange of international best practices and experiences. Uganda has set an ambitious agenda to develop its substantial energy and mineral resources, promote economic development, end energy poverty, and lead the country to a just energy transition. Uganda’s stated objective in Vision 2040 is to transform into “a modern and prosperous country”, ensuring a better future for its citizens. The energy sector will play an important role in helping Uganda achieve this. The newly launched Energy Policy for Uganda 2023 will serve as a crucial tool and major contribution to the country’s ambitious agenda. Uganda already has in place much of the technical expertise, government institutions and policy frameworks to reach its energy goals. It has also made significant progress over the past two decades in providing access to electricity and expanding generation capacity, and further ambition is encouraged to achieve universal energy access by 2040. This report assesses the energy sector and the related challenges facing Uganda and serves as a situational analysis that feeds into the development of the country’s Energy Transition Plan to provide policy recommendations and support the development of the energy sector and the path towards universal access for all.

Abstract: Government action plays a pivotal role in ensuring secure and sustainable energy transitions and combatting the climate crisis. Energy policy is critical not just for the energy sector but also for meeting environmental, economic and social goals. Governments need to respond to their country’s specific needs, adapt to regional contexts and help address global challenges. In this context, the International Energy Agency (IEA) conducts Energy Policy Reviews to support governments in developing more impactful energy and climate policies. This Energy Policy Review was prepared in partnership between the Government of Estonia and the IEA. It draws on the IEA's extensive knowledge and the inputs of expert peers from IEA member countries to assess Estonia’s most pressing energy sector challenges and provide recommendations on how to address them, backed by international best practices. The report also highlights areas where Estonia’s leadership can serve as an example in promoting secure clean energy transitions. It also promotes the exchange of best practices among countries to foster learning, build consensus and strengthen political will for a sustainable and affordable clean energy future.

Abstract: Latin America and the Caribbean is a region that stands out in the global energy sector. It boasts extraordinary natural resources – both fossil fuels and renewable energy – and a significant share of the world’s critical minerals. It also has a history of ambitious policy making in pursuit of stronger energy security and greater sustainability that has delivered one of the cleanest electricity mixes in the world. As the region emerges from a period of sluggish economic growth, countries in Latin America and the Caribbean now stand to leverage these resources to revitalise their economies and improve the security and sustainability of energy around the world. The Latin America Energy Outlook, the International Energy Agency’s first in-depth and comprehensive assessment of Latin America and the Caribbean, builds on decades of collaboration with partners. In support of the region's energy goals, the report explores the opportunities and challenges that lie ahead. It provides insights on the ways in which the outlook for the region and the biggest global energy trends are deeply intertwined – as well as recommendations on policies that could allow Latin America and the Caribbean to take full advantage of its great potential.

Abstract: Governments, businesses and citizens around the globe are facing the challenge of climate change and how to accelerate global clean energy transitions to reach net zero emissions by 2050 at the latest. Central to reducing energy-related emissions are the move away from fossil fuel use to electrified systems; significant and sustained improvements in energy efficiency; and an increase in renewable electricity generation capacity. Japan’s Ministry of Economy, Trade and Industry, as part of Japan’s Presidency of the G7, asked the International Energy Agency (IEA) to examine the evolution of energy efficiency policy in the context of the clean energy transitions. This acted to support discussions among G7 countries to provide insights and direction for the G7 energy and climate agenda. This report reflects the findings of these discussions, and uses them as a basis for guiding policy makers towards the development of policies to hasten the delivery of the energy system required for clean energy transitions. The challenge is considerable but, as this report demonstrates, governments have already begun the process of transforming their energy efficiency policies to address it. The report outlines the ways in which energy efficiency polices can develop to incorporate flexibility and engagement levers and provides insights into policy developments in major economies, with examples in three main sectors: demand flexibility in appliances and buildings, vehicle fuel economy standards, industrial energy and carbon reporting.

Abstract: This dataset includes oil product spot prices by market. Products such as gasoil, gasoline, high sulphur fuel oi, jet kerosene, naphtha are listed within the database.

Abstract: This database comprises various energy product prices expressed in US dollars with products such as sulphur fuel oil, automotive diesel, premium leaded gasoline while comparing data by sectors.

Abstract: The world remains far off track to reach universal access to electricity by 2030 – a key target canonised in UN Sustainable Development Goal 7 (SDG7). While many countries have set targets to advance universal electrification, it remains challenging to collect timely data on their progress, making it difficult to build or refine their electrification plans and policy strategies. To provide governments with the tools to achieve their goals, the Guidebook for Improved Electricity Access Statistics (hereafter “Guidebook”) focuses on methodologies using readily available supply-side data from electric utilities, mini-grid operators, and off-grid system distributors to track access to electricity trends. This approach can be adopted at a low-cost and provides accurate estimates on access rates with a potential time lag on the order of months, instead of years. Supply-side data complements household or census surveys, which can give a more nuanced, detailed picture, but are typically only run every five to ten years due to their expense. This report provides step-by-step guidelines on how to implement an improved supply-side data collection process, and produce more time-sensitive and comprehensive access to electricity indicators. In addition, the Guidebook proposes a standard method to capture mini-grids and stand-alone off-grid systems in calculating access to electricity, which are becoming increasingly more commonplace in strategies to reach universal access. Finally, the Guidebook suggests further steps practitioners can take to advance geospatial data collection and reporting, which has become the new best practice for electrification planning supporting both electric utilities and the vibrant off-grid industry.