Abstract: Responsible Generative AI and Local LLMs Learning Generative AI During Live Coding This video series covers mastering the theory of Generative AI and using local LLMs like Mistral, llamafile, Candle, and Lorax. Lessons covered include: Profit Sharing Concepts Tragedy of the Commons Game Theory and Generative AI Perfect Competition Negative Externalities Fine-tuning Mistral with Ludwig Getting Started with llamafile Hello World in Candle Exploring Rust and Candle Transcribing Audio with Whisper Rust for LLMs Learning Objectives Learn about responsible and ethical uses of generative AI Getting hands-on with local LLMs like Mistral, llamafile, Candle, and Lorax Learning to build solutions by coding along during tutorials Learning the key syntax and features of the Rust Language Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: GenAI and LLMs on AWS Learn to build machine learning pipelines leveraging large language models on AWS for natural language processing and generative AI. In this hands-on course, you will gain skills to: * Set up cloud-based Rust development environments * Construct serverless workflows with AWS Lambda and Step Functions * Generate text and code with models like Claude and CodeWhisperer * Orchestrate models and data workflows with Amazon Bedrock * By completing real-world coding projects, you will be prepared to operationalize LLMs for machine learning applications. Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Interactive Technical Projects A weekly updated series of tutorials on interactive technical projects Learning Objectives Learn practical skills by building real-world projects with cutting-edge technologies Learning to build full stack solutions with languages like Rust, Python, LLMs and Data Science Learning the key tools and techniques used by industry experts to create robust applications Get hands-on practice applying your core technical knowledge Understand how all the pieces fit together by building projects end-to-end Troubleshoot bugs and solve complex problems that appear in production systems Expand your portfolio and r©♭sum©♭ with impressive projects Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Introduction to Generative AI Get started with Generative AI and Large Language Models This introductory course is designed for beginners with no prior knowledge of generative AI. You will start by gaining a high-level understanding of what generative AI is and how it works. Through interactive lessons and hands-on examples, you will learn fundamental skills like providing effective prompts and iteratively improving the generated outputs. As the course progresses, you will dive deeper into specific major generative AI models, including their unique capabilities and limitations. Finally,, you will get practical experience using leading systems like GitHub Copilot, Qdrant, and OpenAI to generate code, and text. By the end, you will have developed core knowledge to start experimenting with generative AI in a responsible and effective way for a variety of applications. This course aims to provide a friendly introduction to prepare complete beginners for further exploration of this rapidly evolving technology. Learning objectives Learn to utilize Generative AI effectively Develop Generative AI software solutions Build solutions with Prompt Engineering to enhance Generative AI output Practice Labs Use the practice labs to test your knowledge and gain hands-on experience working with Python and Rust. These labs are based on GitHub templates that will help you to get started quickly and apply the knowledge in the course in building your own solutions. Learn Retrieval Augmented Generation: https://github.com/alfredodeza/learn-retrieval-augmented-generation Azure Chat with OpenAI: https://github.com/alfredodeza/azure-chat-demo Resources Azure LLMOps Practical MLOps book Rust Bootcamp Python Bootcamp.

Abstract: MySQL for Data Engineering Learning Linux Hacker Style Coding Description: Extract MySQL data through queries and bash pipelines Dump query results to CSV files. Build web services to serve data with Python standard library Analyze and process extracts with Linux utilities Chain MySQL, Python, and Linux together for quick data tasks Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Agile with AI Enhancing Agile Methodologies with Artificial Intelligence This video course explores the integration of Agile practices with Artificial Intelligence (AI) to streamline software development processes and improve productivity. By combining the principles of Agile and the capabilities of AI, you can unlock new levels of efficiency and innovation. Lessons Covered Include: 1.0 Course Introduction 2.1 Waterfall Planning 2.2 Playground vs. Plant Analogy 3.2 Pragmatic Agile 3.15 Productivity Techniques 4.1 AIOps (Artificial Intelligence for IT Operations) 4.2 Diagram Prompt Engineering 4.3 Summarizing Text with Claude 4.4 AI Pair Programming: Paradigm Shift 4.5 AWS CodeWhisperer for Rust 4.7 Serverless Bedrock Diagram 4.8 Bedrock Knowledge Agent: Retrieval-Augmented Generation (RAG) 4.9 Building Blocks for Generative AI: WhisperCPP RAW Learning Objectives Understand the foundations of Agile methodologies and their importance in software development Learn techniques for enhancing Agile practices with AI tools and techniques Explore AI-powered productivity techniques for streamlining development workflows Discover AI-driven approaches to pair programming, code generation, and prompt engineering Gain insights into cutting-edge AI applications in software development, such as AIOps and knowledge agents Learn to leverage AI tools like AWS CodeWhisperer and Retrieval-Augmented Generation (RAG) models Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Small Language Models and LLamaFile Learning About Small Language Models and Running Them Locally with LLamaFile This video series covers small language models, their advantages, and how to run them locally using the LLamaFile tool. Lessons Covered Include: 1.1 Introduction to Small Language Models 1.2 Demo: Running Phi Small Language Model with LLamaFile 2.1 Getting Started with LLamaFile 2.2 Demo: Running Lava Multi-Modal Vision Model with LLamaFile Learning Objectives Understand what small language models are and their advantages Learn how to run local language models using the LLamaFile tool See demos of running different small language models like Phi and Lava Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Introduction to LLM vulnerabilities This introductory course on vulnerabilities for Large Language Models (LLMs) and language models in general. It provides a deep dive into the practical applications of large language models (LLMs) using Azure's AI services. Upon completion, learners will be able to: Explain the concept of model replication or model shadowing as a potential attack vector in large language models, and describe methods to mitigate it through techniques like rate limiting and buffering. Analyze the potential benefits and limitations of using pre-trained LLMs Develop strategies for mitigating risks and ethical considerations when deploying LLM-powered applications. Describe the high-level process of creating a large language model, including data collection, cleaning, and training. Explain the role of security in large language models and recognize potential security vulnerabilities and attack vectors. Identify insecure plugin designs in large language model software development kits (SDKs) that could lead to remote execution and implement strategies to secure plugins. You will learn how to secure your large language model (LLM) applications by addressing potential vulnerabilities. You will explore strategies to mitigate risks from insecure plugin design, including proper input validation and sanitization. Additionally, you will discover techniques to protect against sensitive information disclosure, such as using a redaction service to remove personally identifiable data from prompts and model responses. Finally, you will learn how to actively monitor your application dependencies for security updates and vulnerabilities, ensuring your system remains secure over time. Week 1: Foundations of Language Models This week you will get a brief overview of LLMs and how do they work Learning Objectives Analyze common types of generative applications and their architectures, including multi-model applications, and understand their challenges and benefits. Explain the functioning of a multi-model application, including the role of the framework and specialized machine learning models. Identify the advantages of smaller, specialized models in terms of resource usage, interaction speed, and deployment agility. Compare and contrast different generative AI application types, such as API-based, embedded models, and multi-model applications, and understand their use cases and challenges. Recognize the importance of large language models in various real-world applications, including text-based chatting, customer service, content creation, and daily tasks. Evaluate the benefits and drawbacks of large language models, considering aspects like accuracy, privacy, and potential misuse. Understand the basics of tokenization, indexing, and probability machines in the context of large language models. Describe the high-level process of creating a large language model, including data collection, cleaning, and training. Explain the role of security in large language models and recognize potential security vulnerabilities and attack vectors. Week 2: Language Model Vulnerabilities This week focuses on model-based vulnerabilities that you can explore with prompts. Learning Objectives Explain the concept of model replication or model shadowing as a potential attack vector in large language models, and describe methods to mitigate it through techniques like rate limiting and buffering. Identify and demonstrate insecure output handling in large language models, and understand the potential security threats and attack vectors associated with it. Understand prompt injection and its implications for large language models, including how certain applications define the initial behavior of these models and how to exploit implicit system prompts. Recognize model theft vulnerabilities and understand how handling and access to system components can impact model security, particularly in the context of dynamically loaded models from external sources. Week 3: System vulnerabilities This week you will learn how to deal with environments and system-based vulnerabilities as they relate to LLMs. Learning Objectives Identify insecure plugin designs in large language model software development kits (SDKs) that could lead to remote execution and implement strategies to secure plugins. Explain the potential risks of sensitive information disclosure in large language models and implement measures to redact personal identifiable information using HTTP APIs and regular expressions. Monitor and update dependencies in large language model applications to prevent potential security vulnerabilities and automate the process using tools like GitHub's Dependabot. Evaluate application vulnerabilities based on the programming language and framework, and implement measures to prevent potential security threats. Week 4: Other types of vulnerabilities Learning Objectives Identify potential security threats and vulnerabilities associated with large and small language models. Implement strategies to prevent security situations and guard against making environments more secure. Recognize the concept of excessive agency in large language models and its potential impacts on functionality. Explain the denial of service threat for large language models and describe methods to guard against API misuse.% About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to understand and work with vulnerabilities within language models. Resources Introduction to Generative AI Responsible Generative AI and Local LLMS Practical MLOps book.

Abstract: This video series covers key concepts and technologies for virtualizing and containerizing applications. It introduces the benefits of virtualization, compares virtual machines and containers, and provides a deep dive into working with Docker containers. Through live demonstrations, you'll learn to create and manage virtual machines, pull Docker images, and run containerized applications.

Abstract: This comprehensive course teaches you to leverage the power of cloud computing to develop production-ready machine learning solutions. Starting with the fundamentals of cloud storage, security, and infrastructure as code, you will gain a solid foundation in cloud computing concepts. The course then dives into serverless machine learning, teaching you to build and deploy ML models using popular cloud AutoML platforms like Google AutoML, Amazon SageMaker, Microsoft Azure ML, and CreateML. Through hands-on labs, you'll learn to train custom models with just a few lines of configuration, without needing deep ML expertise. Next, the course explores serverless technologies like AWS Lambda, Google Cloud Functions, and Azure Functions to cost-effectively operationalize your ML models at scale. You'll learn to expose your AutoML models as serverless microservices, automatically scaling to handle any load. The course also covers serverless data processing and ETL to build complete ML pipelines. Finally, you'll learn to apply DevOps best practices to your serverless ML applications using continuous integration and delivery (CI/CD), infrastructure as code, monitoring, and containerization technologies like Docker and Kubernetes. By the end, you'll have the skills to build highly scalable end-to-end machine learning solutions leveraging the latest cloud AutoML and serverless technologies.

Abstract: Hands-on Python for MLOps Build powerful APIs, tools, and scripts with Python Use Python to create APIs and automation scripts for machine learning operations. Learn to leverage APIs, build CLI tools, and develop web services. If you are new to Machine Learning Operations or want to know more about the foundations of Python for MLOps, this course will walk you through the basics. In this course, you'll learn how to use Python for automation, APIs, and building useful command-line interfaces. We'll cover consuming APIs, creating CLI tools, packaging projects, and building web services to expose machine learning models. The course includes hands-on examples and projects so you can apply what you learn right away. By the end of the course, you'll have practical skills to automate workflows and develop interfaces for machine learning. This course includes GitHub repositories you can reference: Python CLI Examples Basic Python CLI Web API with Python and Hugging Face models Learn objectives Consume and use APIs and SDKs in Python Create command-line programs for automation Develop web services and APIs with Python frameworks Package Python projects for distribution Apply skills to build useful interfaces for ML models Lesson 1: Working with APIs in Python Lesson Outline Overview of APIs and SDKs Using the Requests Library Consuming REST APIs Python SDKs like NumPy and SciPy Project: Building a Python script using APIs Lesson 2: Building Command-line Interfaces Lesson Outline Intro to automation with CLI tools Parsing arguments and options Organizing code into modules Python packaging for distribution Project: CLI tool for machine learning Lesson 3: Developing Web APIs Lesson Outline REST API concepts Web frameworks like Flask and FastAPI Building an API with Flask Developing APIs with FastAPI OpenAPI specs and documentation Project: Web API for movie recommendations About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his past experience as a Linux system administrator is seen throughout this course, where you will get a first-hand experience with high-level knowledge and practical examples. Resources Pytest Master Class Practical MLOps book.

Abstract: AWS Certified Security - Specialty Exam Prep This video series provides comprehensive preparation for the AWS Certified Security - Specialty (SCS-C02) exam. Lessons Covered Include: Domain 1: Threat Detection and Incident Response 1.0 Introduction 1.1 Designing and Implementing Incident Response Plans 1.2 Detecting Security Threats and Anomalies with AWS Services 1.3 Responding to Compromised Resources and Workloads 1.4 Automating Incident Response with AWS Lambda 1.5 Conducting Root Cause Analysis with Amazon Detective 1.6 Capturing Forensics Data from Compromised Resources 1.7 Querying Logs to Validate Security Events 1.8 Preserving Forensic Artifacts with S3 Object Lock 1.9 Preparing and Recovering Services After Incidents 1.10 Incident Response Case Study 1.11 Practice Exam - Threat Detection and Response 1.12 Threat Detection and Incident Response Summary Learning Objectives Understand best practices for incident response in the cloud Explore AWS services to detect security threats Learn how to respond to compromised resources Domain 2: Security Logging and Monitoring 2.0 Introduction 2.1 Designing and Implementing Monitoring and Alerting 2.2 Troubleshooting Security Monitoring and Alerting 2.3 Designing and Implementing Logging Solutions 2.4 Troubleshooting Logging Solutions 2.5 Designing Log Analysis Solutions 2.6 Configuring AWS CloudTrail for Logging 2.7 Analyzing Logs with Amazon Athena 2.8 Monitoring Case Study 2.9 Practice Exam - Logging and Monitoring 2.10 Optimizing Log Storage Lifecycles 2.11 Third Party Log Analysis Tools 2.12 Security Logging and Monitoring Summary Learning Objectives Understand how to set up monitoring and alerts for security Learn best practices for logging and log analysis Gain skills in troubleshooting logging and monitoring Domain 3: Infrastructure Security 3.0 Introduction 3.1 Designing and Implementing Edge Service Security 3.2 Designing and Implementing Network Security 3.3 Designing and Securing Compute Workloads 3.4 Troubleshooting Network Security 3.5 Securing EC2 Instances with SSM 3.6 Scanning Images for Vulnerabilities 3.7 Infrastructure Security Case Study 3.8 Practice Exam - Infrastructure Security 3.9 Integrating AWS Firewall Manager 3.10 Applying Security Best Practices to Lambdas 3.11 Hardening AMIs with EC2 Image Builder 3.12 Infrastructure Security Summary Learning Objectives Learn how to secure AWS network infrastructure Understand options for compute and workload security Gain skills in troubleshooting infrastructure issues Domain 4: Identity and Access Management 4.0 Introduction 4.1 Designing Authentication for AWS Resources 4.2 Designing Authorization for AWS Resources 4.3 Troubleshooting with IAM Access Analyzer 4.4 Enforcing Least Privilege Access 4.5 Separating Duties with IAM Roles 4.6 Identity and Access Management Case Study 4.7 Practice Exam - Identity and Access Management 4.8 Securing Root User Credentials 4.9 IAM Policy Simulator for Troubleshooting 4.10 Enabling Federated Access with SSO 4.11 Managing Credentials and Secrets 4.12 Identity and Access Management Summary Learning Objectives Master IAM best practices for authentication and authorization Understand how to apply least privilege and separation of duties Learn to troubleshoot IAM issues Domain 5: Data Protection 5.0 Introduction 5.1 Data in Transit Confidentiality and Integrity 5.2 Data at Rest Confidentiality and Integrity 5.3 Managing Data at Rest Lifecycles 5.4 Protecting Credentials, Secrets, and Keys 5.5 Encrypting Data with AWS KMS 5.6 Data Protection Case Study 5.7 Practice Exam - Data Protection 5.8 Enforcing Data Retention with Glacier 5.9 Preventing Data Deletion with S3 Object Lock 5.10 Rotating RDS Credentials 5.11 Encrypting Data in Transit with SSL/TLS 5.12 Data Protection Summary Learning Objectives Learn techniques to protect data in transit and at rest Understand how to manage data lifecycles Gain skills in managing credentials and secrets Domain 6: Management and Security Governance 6.0 Introduction 6.1 Strategies for Central Account Deployment 6.2 Secure and Consistent Cloud Deployment 6.3 Evaluating Compliance of AWS Resources 6.4 Identifying Security Gaps through Reviews and Cost 6.5 AWS Organizations SCP Best Practices 6.6 Governance Case Study 6.7 Practice Exam - Governance 6.8 Automating Security Deployments 6.9 Tagging Resources for Governance 6.10 AWS Config for Security Assessments 6.11 AWS Well Architected Tool 6.12 Governance and Security Management Summary Learning Objectives Understand best practices for governing accounts and resources Learn how to align security with organizational requirements Gain skills in compliance, auditing, and deployment Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification AWS Solutions Architect Professional (SAP-C02) 2023 Assimilate Python From Zero.

Abstract: Rust Bootcamp Course Overview Welcome to our comprehensive Rust programming course! In this immersive learning experience, we will guide you through every aspect of Rust, empowering you to become a proficient developer. This course is ideal for beginners wanting to become proficient in Rust or from existing developers coming from languages like Python or JavaScript that want to learn Rust fundamentals. Get started by setting up your preferred text editor and installing Rust. We'll ensure that you have all the necessary tools to create a productive coding environment. By leveraging the capabilities of Visual Studio Code and enabling the Rust Analyzer, you'll unlock a feature-rich ecosystem that enhances your development workflow. Experience the game-changing potential of GitHub Copilot, an AI-powered assistant. Sign up for GitHub Copilot and witness its transformative impact on your coding experience. We'll guide you through the installation process and show you how to leverage its intelligent suggestions for accelerated programming. Master concepts and fundamentals like control flow, variable assignment, and immutability. Learn the intricacies of loops, conditional statements, and effective error handling techniques. With Rust's borrowing concept at your disposal, you'll develop secure and high-performance code. Build upon your foundation with advanced topics like structs, string manipulation, and working with vectors and enums. We'll explore real-world library development using Cargo, Rust's robust package manager. Effective documentation and debugging techniques will become second nature as you optimize your code. Organize your code effectively with modules, extending functionality and enhancing reusability. Dive into comprehensive testing methodologies to ensure the reliability and correctness of your programs. From writing tests for your code to handling private components, you'll gain expertise in developing robust software projects. Our immersive learning experience includes hands-on examples, interactive exercises, and practical projects. Rust's performance, safety, and expressiveness will be at your fingertips as you embrace its immense potential. Join us on this rewarding journey to unlock the power of Rust programming and propel your career forward. Enroll now and gain the skills, knowledge, and expertise to thrive in the world of Rust. Whether you're a beginner, coding novice, or an experienced developer seeking new horizons, our course is your gateway to writing elegant, reliable, and high-performance code. Embrace the possibilities of Rust and embark on a rewarding programming journey today with one of the most loved programming languages. Learning Objectives Gain a solid understanding of Rust's core concepts, including variable assignment, control flow, and immutability, to write efficient and reliable code. Master the utilization of powerful development tools such as Visual Studio Code and the Rust Analyzer, enhancing productivity and enabling seamless coding experiences. Explore GitHub Copilot and learn how to leverage its AI-driven suggestions to accelerate programming and boost code quality. Develop proficiency in working with advanced Rust features, such as structs, enums, and string manipulation, to build robust and flexible applications. Discover the intricacies of module organization and extend functionality through effective code structuring, promoting code reusability and maintainability. Acquire comprehensive testing skills, including writing tests for code components and handling private code, ensuring the reliability and correctness of Rust programs. Gain hands-on experience in real-world library development using Cargo, Rust's package manager, and become proficient in documenting code and debugging techniques. These compelling learning objectives will guide you through the course, helping you acquire essential skills and knowledge to become a confident Rust developer. Course Content This course is divided into 4 weeks with 3 lessons that contain about 6 videos each. Each video is about 5 minutes long and contains a hands-on demonstration of the concepts covered in the lesson. The course is designed to be taken in order, but you can jump to any lesson you want to learn more about. Week 1: Setting up your development environment Reference GitHub Repository Setting up your text editor Using GitHub Copilot with Rust Introduction to GitHub Codespaces for Rust Week 2: Rust fundamentals Reference GitHub Repository Introduction to Rust Loops and Control flow Function basics Week 3: Structs, Types, and Enums Reference GitHub Repository Using structured data Exploring strings and vectors Working with Enum and Variants Week 4: Applying Rust Reference GitHub Repository Building a real-world library Extending functionality with modules Testing Rust code About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to build with the Rust programming language. Resources Deploy Rust on Azure Functions DevOps command-line tools in Python and Rust Switching to Rust from Python First Steps with Rust Learning Path.

Abstract: Live Coding Rust Candle from Hugging Face Learning LLMOps This video series covers live coding Rust Candle from Hugging Face for LLMOps Lessons Covered Include: 1.0 Building binaries for Phi using AWS CodeBuild and Rust Candle Learning Objectives Run your own Rust LLMs Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: GitHub Enterprise - Certified Learning the GitHub Enterprise Certification Material This course covers all seven domains of the the GitHub Enterprise Certification Lessons Covered Include: Domain 1: Support GitHub Enterprise for users and key stakeholders Distinguish admin tasks vs needing GitHub Support Generate support bundles and diagnostics Identify underutilized features, integrations, active teams/repos Recommend standards for workflows, branching, code review, automation Explain tooling ecosystem and CI/CD strategy Recommend workflows and tooling to teams Use GitHub APIs to extend admin capabilities Find Marketplace assets for specific needs Contrast GitHub Apps vs actions Domain 2: Manage user identities and GitHub authentication Implications of enabling SAML SSO for orgs/enterprise Steps to enable and enforce SAML SSO Require two-factor authentication Supported identity providers GitHub identity management and authorization Authentication and authorization model Supported SCIM providers How SCIM and team sync works Domain 3: Describe deployment, distribution, licensing Capabilities of GHES, GHEC, GHAE Billing models including licenses, Actions, Packages License usage statistics for orgs, enterprise, machine accounts Consumption of metered products Domain 4: Manage access and permissions Enterprise permissions and policies Organization permissions Team permissions Repository permissions Domain 5: Enable secure development and compliance How GitHub supports enterprise security Scrubbing sensitive data Choosing policies for control required Impacts of policies Organization and enterprise policies Use audit log APIs to explain missing assets Security features of GitHub repositories Create security response plan SSH keys and deploy keys Domain 6: Manage GitHub Actions Distribute actions and workflows in enterprise Manage runners Manage encrypted secrets Domain 7: Manage GitHub Packages Supported Packages Accessing, writing, and sharing Packages Using Packages in workflows Contrasting Packages and releases Learning Objectives Manage user access and permissions across GitHub Enterprise environments through authentication, organizations, teams, and repositories. Establish secure development practices and compliance controls with GitHub features. Distribute and manage GitHub Actions across the enterprise. Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Hugging Face for MLOps Learn how to leverage Hugging Face and its powerful machine learning capabilities. Build, train, and deploy your own models using the Hugging Face platform and libraries. In this course you'll get hands-on with Hugging Face and learn how to: Access and use pre-trained models from the Hub Fine-tune models with your own data Build machine learning pipelines with Hugging Face Transformers Add and version your own datasets Containerize and deploy Hugging Face models Automate workflows with GitHub Actions By the end, you'll have practical experience building, training, and deploying Hugging Face models, including production deployment to the Azure cloud. Learning objectives Find and use pre-trained models Fine-tune models for custom tasks Build ML pipelines with Hugging Face libraries Create and version datasets Containerize models for production Automate workflows for MLOps Lesson 1: Getting Started with Hugging Face Lesson Outline Overview of Hugging Face Hub Browsing models and datasets Using Hugging Face repositories Managing spaces and access Lesson 2: Applying Hugging Face Models Lesson Outline Downloading models from the Hub Using models with PyTorch/TensorFlow Leveraging tokenizers and pipelines Performing inference with Hub models Lesson 3: Working with Datasets Lesson Outline Browsing datasets on Hugging Face Uploading and managing datasets Versioning datasets with dataset cards Loading datasets in PyTorch/TensorFlow Lesson 4: Model Serving and Deployment Lesson Outline Containerizing Hugging Face models Creating inference APIs with FastAPI Deploying to cloud services like Azure Automating with GitHub Actions About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his experience teaching and authoring content about LOps will give you everything you need to get started applying these powerful concepts. Resources MLOps with Databricks Introduction to MLflow for MLOps Hands-on Python for MLOps.

Abstract: Rust AWS Lambda Build and deploy serverless applications with Rust and AWS Lambda This course will help you build and deploy serverless applications with Rust and AWS Lambda. You will learn how to create, deploy, and manage AWS Lambda functions using Rust, leveraging the power and performance of the language for serverless architecture, and apply it by building a real-world Lambda function as a part of a distributed application. What you will learn Understand the concept of serverless architecture and AWS Lambda. Create and deploy AWS Lambda functions written in Rust. Integrate Lambda functions with other AWS services. Monitor, test, and optimize Rust-based Lambda functions. Implement real-world applications using Rust and AWS Lambda. Lesson 1: Getting Started with Rust AWS Lambda and Serverless Introduction to Serverless and AWS Lambda Walk through Rust Firecracker Project Setting up VSCode AWS Toolkit and CodeWhisperer for Rust Introduction to Cargo Lambda Rust Cost Advantage AWS Lambda Using the AWS Lambda Console Using the Step Functions Console Invoking Step Functions from AWS CLI Lesson 2: Advanced Techniques with Rust and AWS Lambda Building a Rust AWS Lambda Add Function Building a Rust AWS Lambda Divide by Two Function Invoking AWS Step Function from CLI Building Chainable AWS Step Functions with Rust Serverless MLOPs with EFS mounted in AWS Lambda Using Rust AWS Lambda Function URLs Saving money with arm64 Rust AWS Lambdas Lesson 3: Building a Polars Rust AWS Lambda Solution Polars Rust AWS Lambda Build and Deploy Polars Rust AWS Lambda Using AWS Console with deployed Polars Rust AWS Lambda Analyzing Rust AWS Lambda code with CodeWhisperer and AWS Toolkit AWS Lambda Function URLs Build and Deploy Polars Rust AWS Lambda Function URLs.

Abstract: Rust LLMOps: A Rough Draft Course (Updated JIT) Learning During Live Coding This video series is a rough draft course being updated JIT (Just in Time) and covers the Rust language through live coding. We offer an iterative approach to understand and master the language, particularly focusing on its application in MLOps. Lessons Covered Include: Rust Hugging Face Candle Description This week, you will delve into the powerful combination of Rust with Candle, a minimalist ML framework, and explore how they can be used with Hugging Face's popular transformer models. You will apply these concepts by working on a series of hands-on labs that guide you through building, training, and deploying machine learning models using Rust, Candle, and Hugging Face. Learning Objectives By the end of this lesson, you will be able to: - Implement Candle in Rust to create and train machine learning models. - Utilize the various features of Candle, including syntax, GPU efficiency, and browser deployment. - Integrate Hugging Face transformers with Rust for natural language processing tasks. - Apply the principles of Rust and Candle to build real-world machine learning applications. - Evaluate the performance and scalability of Rust-based ML solutions using Candle and Hugging Face. Lesson Structure Original 4.1 Candle: A Minimalist ML Framework for Rust 4.2 Using GitHub Codespaces for GPU Inference with Rust Candle 4.3 VSCode Remote SSH development AWS Accelerated Compute 4.4 Building Hello World Candle 4.5 Exploring StarCoder: A State-of-the-Art LLM for Code 4.6 Using Whisper with Candle to Transcribe 4.7 Exploring Remote Dev Architectures on AWS.

Abstract: Rust For DevOps Learn how to use and apply DevOps principles using Rust This is an early release of the course. In this course, you will learn the basics of DevOps using the Rust programming language. DevOps helps you build, test, and deploy code faster while creating a robust foundation for your projects and teams. First, we will talk about what DevOps is. You will learn things like automation, monitoring, and working together as a team. Then, we will see how to use DevOps ideas in Rust code. You will learn about version control, testing, containers, and deploying Rust programs. Although this course explains the foundations of DevOps and using Rust which is ideal for beginners, experienced Rust developers will find solid advice in how to apply DevOps best practices to existing applications. Learning Objectives The core principles and practices of DevOps How to apply DevOps concepts to build, package, and deploy applications with Rust The basics of containerization and how to leverage containers for DevOps with Rust Course Content This course is divided into 4 weeks with 3 lessons that contain about 6 videos each. Each video is about 5 minutes long and contains a hands-on demonstration of the concepts covered in the lesson. The course is designed to be taken in order, but you can jump to any lesson you want to learn more about. Week 1: Introduction to DevOps Principles Introduction to DevOps Principles DevOps Considerations for Applications Basics of Containerization Week 2: Implementing monitoring and logging strategies Introduction to monitoring and logging strategies Tools and Services for monitoring and logging Logging with Rust About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to build robust applications and infrastructure with the Rust programming language and for Rust applications. Resources Rust Bootcamp Deploy Rust on Azure Functions DevOps command-line tools in Python and Rust Switching to Rust from Python First Steps with Rust Learning Path.

Abstract: Learning Generative AI with AWS CodeWhisperer During Live Coding This video series covers live coding in various languages iteratively, thus learning the language. Lessons Covered Include 1.0 Getting started with Bash and Rust using AWS CodeWhisperer, VSCode, and AWS Toolkit for VSCode Learning Objectives Learn the fundamentals of Generative AI Learn to build AI-powered solutions with Bash, Rust, and AWS services Master the key syntax and features of Bash and Rust languages Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Rust for Pythonistas What you'll learn-and how you can apply it A multi-hour jam session where you will learn to code in Rust. * Write efficient and safe Rust code * Leverage Rust libraries and tools in your projects * Combine Rust with Python for improved application performance Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: GitHub Foundations: Privacy, Security and Administration (Rough Draft) Updated (JIT) as it is built About This Video Course Building Secure and Efficient GitHub Workflows This video series covers live coding the GitHub Administration and Security domain iteratively, thus learning the best practices and nuances of managing and securing GitHub projects. Lessons Covered Include: 1.0 Course Introduction: Building Secure and Efficient GitHub Workflows 1.1 Introduction to Authentication and Security: Ensuring Your GitHub Account's Integrity 1.2 Securing Account with 2FA 1.3 Understanding Access Permissions 1.4 Enterprise Managed Users (EMUs) Explained 1.5 Introduction to GitHub Administration: Configuring and Managing Repositories and Organizations 1.6 Enabling and Disabling Features 1.7 Repository Permission Levels Explained 1.8 Setting Repository Visibility Options 1.9 Repository Privacy Settings and Options 2.0 Features and Options in the Security Tab 2.1 Understanding Repository Insights 2.2 Managing Collaborators in GitHub 2.3 Managing Organization Settings 2.4 Members, Teams, and Roles in a GitHub Organization 2.5 Next Steps: Continuing Your GitHub Mastery Journey Learning Objectives Learn GitHub's authentication and security measures Learning to build solutions with robust repository management Learning the key syntax and features of GitHub Administration and Security Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification AWS Solutions Architect Professional (SAP-C02) 2023 Assimilate Python From Zero.

Abstract: Live Coding in Rust: Unleash the Power of Systems Programming Delve deep into the Rust ecosystem with this groundbreaking live coding series, focused on building robust and performant software, including cutting-edge topics like LLMOps and machine learning with Hugging Face's Candle library. What You'll Get Live coding sessions with expert commentary Weekly updates and continually expanding! High-impact projects and examples Exclusive discussions on Rust best practices Topics covering LLMOps, machine learning, and more A focus on performance, safety, and systems programming Course Outline (Future and Current Content) Rust Fundamentals Introduction to Rust Understanding Ownership & Borrowing Error Handling in Rust LLMOps in Rust Live Coding Rust: LLMOps in Rust--Exploring Hugging Face Candle Building Custom LLMOps Libraries Rust and LLVM: A Match Made in Heaven Machine Learning in Rust Implementing ML Algorithms in Rust Using Hugging Face Candle for NLP Deploying Rust ML Models in Production Systems Programming Writing File Systems in Rust Creating Web Servers with Actix Asynchronous Programming with async/await Rust in DevOps Automating CI/CD Pipelines with Rust Rust for Cloud Operations Rust in Containerized Environments Topics Covered LLMOps : The intersection of LLVM and DevOps Machine Learning : Building and deploying models in Rust Systems Programming : Low-level operations made easier and safer DevOps : Leverage Rust for automating operations tasks Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Rust Data Engineering: Course Description Are you a data engineer, software developer, or a tech enthusiast with a basic understanding of Rust, seeking to enhance your skills and dive deep into the realm of data engineering with Rust? Or are you a professional from another programming language background, aiming to explore the efficiency, safety, and concurrency features of Rust for data engineering tasks? If so, this course is designed for you. While a fundamental knowledge of Rust is expected, you should ideally be comfortable with the basics of data structures and algorithms, and have a working understanding of databases and data processing. Familiarity with SQL, the command line, and version control with git is advantageous. This four-week course focuses on leveraging Rust to create efficient, safe, and concurrent data processing systems. The journey begins with a deep dive into Rust's data structures and collections, followed by exploring Rust's safety and security features in the context of data engineering. In the subsequent week, you'll explore libraries and tools specific to data engineering like Diesel, async, Polars, and Apache Arrow, and learn to interface with data processing systems, REST, gRPC protocols, and AWS SDK for cloud-based data operations. The final week focuses on designing and implementing full-fledged data processing systems using Rust. By the end of this course, you will be well-equipped to use Rust for handling large-scale data engineering tasks, solving real-world problems with efficiency and speed. The hands-on labs and projects throughout this course will ensure you gain practical experience, putting your knowledge into action. This course is your gateway to mastering data engineering with Rust, preparing you for the next level in your data engineering journey. Learning Rust During Live Coding This video series covers live coding the Rust language iteratively, thus learning the language. Lessons Covered Include: Section 1: Rust Data Structures: Collections Lesson 1: Getting Started With The Modern Rust Development Ecosystem Meet the instructor & Course Overview: 1.0-meet-instructor-course-overview.mp4 Introduction to the AI Coding Paradigm Shift: 1.1-ai-pair-programming-paradigm-shift.mp4 Introduction to cloud-based development environments: 1.2-GitHub-Codespaces-Ecosystem-with-copilot-chat.mp4 Introduction to GitHub Copilot Ecosystem for Rust: 1.3-copilot-enabled-rust.mp4 Prompt Engineering with GCP BigQuery SQL: 1.4-big-query-prompt-engineering.mp4 Introduction to AWS CodeWhisperer for Rust: 1.5-aws-codewhisperer-for-rust.mp4 Using Google Bard to Enhance Productivity: 1.6-using-bard-to-enhance-productivity.mp4 Continuous Integration with Rust and GitHub Actions: 1.7-continuous-integration-rust-github-actions.mp4 Lesson 2: Rust Sequences and Maps Introducing Rust Sequences and Maps: 1.8-rust-sequences-maps.mp4 Print Rust data structures demo: 1.9-Print-Rust-data-structures-demo.mp4 Vector Fruit Salad demo: 1.10-Vector-Fruit-Salad-demo.mp4 VecDeque Fruit Salad demo: 1.11-VecDeque-Fruit-Salad-demo.mp4 Linkedin List Fruit Salad demo: 1.12-Linkedin-List-Fruit-Salad-demo.mp4 Fruit Salad CLI demo: 1.13-Fruit-Salad-CLI-demo.mp4 HashMap frequency counter demo: 1.14-HashMap-frequency-counter-demo.mp4 HashMap language comparison: 1.15-HashMap-language-comparison.mp4 Lesson 3: Rust Sets, Graphs and Miscellaneous Data Structures Analyzing UFC Fighter Network Using Graph Centrality in Rust: 1.16-ufc-graph-centrality.mp4 Storing Unique Fruits Using HashSet in Rust: 1.17-unique-fruits-with-hashset.mp4 Maintaining Sorted and Unique Fruits Using BTreeSet in Rust: 1.18-sorted-unique-fruits-with-btreeset.mp4 Creating a Fig Priority Fruit Salad Using Binary Heap in Rust: 1.19-fig-priority-fruit-salad-with-binary-heap.mp4 PageRank algorithm for sports data: 1.20-pagerank-sports.mp4 Showing shortest path with dijkstra: 1.21-shortest-path.mp4 Detecting Strongly Connected Components: A Deep Dive into Kosaraju's Algorithm: 1.22-strongly_connected_components_with_kosaraju.mp4 Simple Charting of Data Structures in Rust: 1.23-ascii-graphing.mp4 Section 2: Safety, Security and Concurrency with Rust Lesson 1: Rust Safety and Security Features Multi-Factor Authentication: 2.1_Multi-Factor_Authentication.mp4 Network Segmentation: 2.2_Network_Segmentation.mp4 Least Privilege Access: 2.3_Least_Privilege_Access.mp4 Encryption: 2.4_Encryption.mp4 Mutable fruit salad: 2.5-mutable-fruit-salad.mp4 Customize fruit salad with a CLI: 2.6-customize-csv-fruit-salad.mp4 Data Race example: 2.7-data-race.mp4 Lesson 2: Security Programming with Rust High Availability: 2.10-high-availability.mp4 Understanding the Homophonic Cipher: A Cryptographic Technique: 2.11-homphonic-cipher.mp4 Decoding the Secrets of the Caesar Cipher: 2.12-caesar-cipher.mp4 Building a Caesar Cipher Command Line Interface: 2.13-caesar-cipher-cli.mp4 Creating a Decoder Ring: A Practical Guide: 2.14-decoder-ring.mp4 Detecting Duplicates with SHA-3: A Data Integrity Tool: 2.15-sha3-dupe-detector.mp4 Incident Response: 2.16-incident-response.mp4 Compliance: 2.17-compliance.mp4 Lesson 3: Concurrency with Rust Core Concepts in Concurrency: 2.20-core-concepts-concurrency.mp4 Dining Philosophers: 2.21-dining-philosopher.mp4 Web Crawl Wikipedia with Rayon: 2.22-web-crawl-wikipedia-rayon.mp4 Intelligent Chatbot with Tokio: 2.23-tokio-chatbot.mp4 Multi-threaded deduplication with Rust: 2.24-data-eng-with-rust-dedupe.mp4 Energy Efficiency Python vs Rust: 2.25-energy-efficiency-python-rust.mp4 Concurrency Stress test with a GPU: 2.26-building-cuda-enabled-stress-test-with-rust-pytorch.mp4 Host Efficiency Serverless Optimization problem: 2.27-host-efficiency-optimization-problem.mp4 Section 3: Rust Data Engineering Libraries and Tools Lesson 1: Using Rust to Manage Data, Files and Network Storage Process CSV files in Rust: 2.31-process-csv-rust.mp4 Using Cargo Lambda with Rust: 2.33-cargo-lambda-rust.mp4 List files on AWS EFS with Rust: 2.34-rust-efs-lister.mp4 Use AWS S3 Storage: 2.35-use-s3-storage.mp4 Use AWS S3 Storage from Rust: 2.36-use-rust-for-s3-storage.mp4 Write encrypted data to tables or Parquet files: 2.37-Write-encrypted-data-to-tables-or-Parquet-files.mp4 Lesson 2: DataFrames with Rust, Python and Notebooks What is Colab?: 2.40-what-is-colab.mp4 Using Bard to enhance notebook development: 2.41-using-bard-to-enhance-productivity.mp4 Exploring Life Expectency in a Notebook: 2.42-life-expectancy-eda.mp4 Load a DataFrame with sensitive data: 2.43-Load-a-DataFrame-with-sensitive-information.mp4 Using MLFlow with Databricks Notebooks: 2.44-mlops-mlflow-tracking.mp4 End to End ML with MLFlow and Databricks: 2.45-end-to-end-ml-with-mlflow-and-databricks.mp4 Comparing DataFrame Libraries between Rust and Python: 2.46-comparing-dataframe-libs-rust-python.mp4 Lesson 3: Data Engineering Libraries and Tools with Rust Parquet file writing and reading with Rust: 2.50-parquet-read-write-rust.mp4 Arrow & Parquet in Rust: 2.51-arrow-parquet-rust.mp4 Serverless functions with Rust and AWS Lambda: 2.52-serverless-rust-aws-lambda.mp4 Polars library overview: 2.53-polars-library-overview.mp4 Building RESTful APIs with Rocket: 2.54-building-restful-apis-rocket.mp4 Utilizing Async Rust in Web Development: 2.55-utilizing-async-rust-web-development.mp4 Applying Data Cleaning Techniques with Rust: 2.56-applying-data-cleaning-rust.mp4 Deploying Rust Applications in a Kubernetes Environment: 2.57-deploying-rust-apps-kubernetes.mp4 Section 4: Designing Data Processing Systems in Rust Lesson 1: Getting Started with Rust Data Pipelines (Including ETL) Jack and the Beanstalk Data Pipelines: 4.1-jack-beanstalk-building-data-pipelines.mp4 Open Source Data Engineering - Pros and Cons: 4.2-open-source-de-pro-con.mp4 Core Components of Data Engineering Pipelines: 4.3-core-components-data-engineering-pipelines.mp4 Rust AWS Step Functions Pipeline: 4.4-rust-aws-step-functions.mp4 Rust AWS Lambda Async S3 Size Calculator: 4.5-rust-async-s3-size-calculator-lambda.mp4 What is Distroless: 4.6-what-is-distroless.mp4 Demo Deploying Rust Microservices on GCP: 4.7-demo-build-deploy-rust-microservice-cloud-run.mp4 Lesson 2: Using Rust and Python for LLMs, ONNX, Hugging Face, and PyTorch Pipelines Introduction to Hugging Face Hub: 4.10-intro-hugging-face-hub.mp4 Rust PyTorch Pre-trained Model Ecosystem: 4.11-rust-pytorch-pretrained-models-ecosystem.mp4 Rust GPU Hugging Face Translator: 4.12-rust-gpu-hugging-face-translator.mp4 Rust PyTorch High-Performance Options: 4.13-high-performance-pytorch-rust-demo.mp4 Rust CUDA PyTorch Stress Test: 4.14-building-cuda-enabled-stress-test-with-rust-pytorch.mp4 EFS ONNX Rust Inference with AWS Lambda: 4.15-efs-onnx-lambda-rust-inference-mlops.mp4 Theory behind model fine-tuning: 4.16-intro-fine-tuning-theory.mp4 Doing Fine Tuning: 4.17-doing-fine-tuning.mp4 Lesson 3: Building SQL Solutions with Rust, Generative AI and Cloud Selecting the correct database on GCP: 4.20-gcp-optimize-d...

Abstract: Using Rust with Python A Practical Guide to Rust for Python Developers Topics: Lesson 1: Introduction to Rust and Python Integration & PyO3 Basics Video 1.1 : Introduction to Rust and Python Video 1.2 : PyO3 Installation Video 1.3 : Basic Rust Library Video 1.4 : Rust to Python Video 1.5 : Rust Ownership Model Video 1.6 : Diagram PyO3 Project Video 1.7 : Python Calc CLI Video 1.8 : PyO3 Features Video 1.9 : PyO3 Exceptions Lesson 2: Advanced Rust and Python Mixing Techniques Video 1.10 : Call Python from Rust Video 1.11 : Run Python Embedded in Rust Video 1.12 : Embedded Rust CLI Diagram Video 1.13 : Embedded Rust CLI Video 1.14 : Embedded Rust CLI Test Video 1.15 : Rust-built Python Tools Video 1.16 : Using Rust Ruff Linter Lesson 3: Python and Rust Command-Line Tools & Recap Video 1.17 : Using Polars with Python and Rust Video 1.18 : Polars CLI in Rust Video 1.19 : Polars CLI Test in Rust Video 1.20 : Polars-CLI Integration Test Video 1.21 : Polars Criterion Benchmarking Learning Objectives Understand the need for Rust and Python integration. Explore the PyO3 library and its usage. Understand the Rust ownership model and its usage in PyO3. Who Should Take This Course Python developers who want to learn Rust and its integration with Python. Rust developers who want to learn how to integrate Rust with Python. Developers who want to learn how to build command-line tools using Rust and Python.

Abstract: Achieving Scalability with Vector, Graph, and Key/Value Databases Description This week we explore vector and graph databases, powerful tools for managing and extracting insights from large, complex datasets. As data volumes continue to grow, scalability is crucial. We'll learn how vector and graph databases can efficiently store data while maintaining relationships, enabling more advanced analytics. Through real-world examples, you'll see how these databases unlock scalability for machine learning, fraud detection, social networks, and more. Learning Objectives Learn Vector, Graph, SQLand Key/Value Databases Learning to build solutions with Rust, Vector, Graph, and Key/Value Databases Learning advanced data engineering Additional Popular Resources 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Databricks Certified Data Engineer Associate Course 1: Databricks Lakehouse Platform Description Learn foundational Databricks capabilities including compute, storage, notebooks, and jobs to build scalable data solutions. Learning Objectives Create clusters and configure runtime environments Perform exploratory analysis with notebooks Schedule and monitor multi-task workflows Course 2: Databricks SQL Description Master Spark SQL for reading, transforming, and loading data at scale. Learn techniques like data validation, custom business logic, and slowly changing dimensions. Learning Objectives Query data in notebooks with Spark SQL Handle complex data types Apply data quality rules Implement slowly changing dimensions Course 3: Databricks ML Description Build ML models with Python and Scala APIs in Databricks. Learn best practices for feature engineering, hyperparameter tuning, and model evaluation. Learning Objectives Engineer features from raw data Tune models with cross validation Evaluate model performance Operationalize models with MLflow Course 4: Databricks Data Engineering Description Architect reliable and performant data infrastructure with Delta Lake, streaming, and autoscaling. Learning Objectives Implement ACID transactions Build streaming ETL solutions Autoscale infrastructure to meet SLAs Migrate data warehouses to lakehouse Course 5: Workloads with Jobs Description Orchestrate workloads using multi-task Jobs with configurable scheduling, dependencies, and error handling. Learning Objectives Schedule notebooks, jobs and pipelines Set dependencies across tasks Handle and retry failures Monitor runs using the Jobs UI Course 6: Data Access with Unity Catalog Description Provide governed data access across storage like ADLS, S3, and GCS using Unity Catalog. Learning Objectives Deploy a Unity Catalog Manage credentials securely Apply object-level security Query data from storage tiers Additional Popular Resources Assimilate OpenAI 52 Weeks of AWS-The Complete Series Microsoft Azure Fundamentals (AZ-900) Certification Rust Bootcamp Python Bootcamp Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Rust Data Engineering Building with the GitHub EcoSystem: Copilot, CodeSpaces, and GitHub Actions Microsoft Azure Fundamentals (AZ-900) Certification Google Professional Cloud Architect Certification Course 2023 (Rough Draft) AWS Solutions Architect Professional (SAP-C02) 2023.

Abstract: Deploy Rust on Azure Functions Learn how to deploy Rust functions to Azure Functions with this comprehensive video course. You will start by creating a simple HTTP-triggered function, and then use GitHub Actions to automate your deployment. Deploying Rust in Azure can be frustrating because there is no first-class support yet for it. You will learn what are some of the caveats and what are some strategies you can use to properly deploy a very basic Rust HTTP API. Course Overview We will start by covering the basics of Azure Functions and Rust, and then dive into more advanced topics such as configuring your function's environment, application settings, and using external crates. By the end of this course, you will have a working knowledge of how to deploy and manage your Rust functions on Azure Functions using automation to quickly re-deploy or update your function. Course Content Introduction to Azure Functions - Learn the basics of Azure Functions and how they work with Rust. Creating Your First Azure Function in Rust - Create a simple HTTP-triggered function in Rust and deploy it to Azure Functions. Configuring Your Function Environment - Learn how to use environment variables to configure your function's behavior. Use GitHub Actions for automation About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. With Alfredo's guidance, you will gain the knowledge and skills needed to deploy and manage your Rust functions on Azure Functions. Resources Deploying containers to Azure Practical MLOps book Automated Azure Resource Cleanup Azure Fundamentals Certification.

Abstract: Google Professional Machine Learning Engineer Course 2023 (Rough Draft) Certification Exam Guide Welcome to the Google Professional Machine Learning Engineer Course! This course is designed to help you prepare for the Google Professional Machine Learning Engineer certification exam. Learning Objectives Develop a deep understanding of Google Cloud technologies and various ML models and techniques to design, build, and productionize machine learning solutions that address specific business challenges while adhering to responsible AI practices. Collaborate effectively with cross-functional teams, including application developers, data engineers, and data governance professionals, to ensure the long-term success of ML models throughout their development, deployment, and maintenance. Master the skills required to design, implement, and manage ML architectures, data pipelines, and metric interpretations, as well as optimize model performance through training, retraining, deploying, scheduling, monitoring, and refining models in scalable and efficient ways. Learn how to use the Google Cloud Platform (GCP) to build and deploy ML models, including how to use GCP services such as BigQuery, Cloud Storage, Cloud AI Platform, and Cloud Functions to build and deploy ML models. Who Should Take This Course? Data scientists Data engineers Machine learning engineers Software engineers Data analysts Data architects Business analysts Anyone interested in learning about machine learning and Google Cloud Platform Course One: Framing ML Problems Course Two: Architecting ML solutions Course Three: Designing data preparation and processing systems Course Four: Developing ML models Course Five: Automating and orchestrating ML pipelines Course Six: Monitoring, optimizing, and maintaining ML solutions Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Assimilate ONNX Learning about ML Model portability During Live Coding This video series covers live coding the Language iteratively thus learning the language. Lessons Covered Include: 1.0 Getting started with ONNX. In this live coding session I was able to get both resnet and ONNX working in Rust with Sonos/Trac. Learning Objectives Learn to work with ONNX models Learning to build solutions with ONNX models in Rust Learning the key syntax and features ONNX models Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Getting Started with the Zig Programming Language Learning the Language During Live Coding This video series covers the basics of the Zig programming language through live coding, allowing viewers to learn the language iteratively as it is being used. Lessons Covered 1.0 Getting Started Learning Objectives Learn the fundamentals of the Zig programming language Learn to build solutions with Zig Learn the key syntax and features of Zig Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Google Professional Cloud Architect Certification Course 2023 (Rough Draft) Welcome to the Google Professional Cloud Architect Certification Course! This course is designed to help you prepare for the Google Cloud Certified Professional Cloud Architect certification exam. As a professional cloud architect, you will be responsible for designing, developing, and managing secure, scalable, and highly available cloud solutions using Google Cloud Platform. Course Overview Course One: Designing and planning a cloud solution architecture 1.0 Rough Draft Introduction Introduction to Google Professional Architect Course Onboarding to GCP Cloud Designing a solution infrastructure that meets business requirements Designing a solution infrastructure that meets technical requirements Designing network, storage, and compute resources Using Google Cloud Functions Running Minikube Running Minikube with FastAPI for Kubernetes development Getting Started with Google Cloud Run Load testing with locust Design technical solutions for evolution and migration SRE Mindset Conclusion and Next Steps Course Two: Managing and provisioning a solution infrastructure Introduction to Google Professional Architect Course Two Exploring Network Topologies in GCP: A Demonstration Embracing Hybrid Networking in Google Cloud Platform Overview of GCP Storage Demo of using the SDK and CLI to manage storage Data lifecycle management Data security Demo of building Rust deduplication finder Compute volatility configuration (preemptible vs. standard) Demo Build and Deploy Rust Actix Microservice Cloud Run Container orchestration with GKE Demo of extending Google Cloud Functions Conclusion Course Two Course Three: Designing for Security and Compliance Introduction to Google Professional Architect Course Three Mastering Identity and Access Management in GCP Data Security Best Practices for GCP Secrets Manager API: A Hands-on Guide Exploring a data poisoning attack Meeting Industry Certification Requirements in GCP Auditing GCP Security with Log Analysis Creating Custom Log Dashboards for GCP Security Monitoring Scanning Web Applications with GCP's Cloud Web Security Scanner Analyzing GCP Logs with BigQuery GCP Security: Conclusion and Next Steps Course Four: Designing and optimizing technical and business processes Introduction to Google Professional Architect Course Four Continuous Delivery: An Introduction Hands-on with Continuous Integration in GCP Troubleshooting Techniques: The Five Whys Implementing Simple GitHub Actions for CI/CD The Three Most Important Files in a Python Project Utilizing Trello for Effective Project Management Leveraging Spreadsheets for Project Management Exploring the MLOps Maturity Model Building Bespoke Systems for Core Business Functions Data Science on Windows: Virtualenv and Pip Site Reliability Engineering (SRE) in GCP Performing Load Testing in GCP: A Demo Course Five: Managing and optimizing cloud solution Implementation Introduction to Google Professional Architect Course Five Makefiles Made Easy: A Comprehensive Guide Configuring Your Bashrc for an Optimal Environment Implementing Continuous Integration for Rust with GitHub Actions Unit Testing in Rust: A Hands-on Demonstration Supercharging Rust Development with Copilot Setting Up a GCP Workstation for Python Development Mastering Google Cloud Shell: A Practical Guide Streamline Your Development with Google Cloud Editor Getting Started with the Google Cloud CLI SDK Unleashing the Power of Google Cloud's 'gcloud' Command-Line Tool Deploying Rust Applications to App Engine: A Step-by Course Six: Ensuring solution and operations reliability Introduction to Google Professional Architect Course Six Monitoring and Logging Rust App Engine Applications: A Demo Uncovering the Advantages of Cloud Developer Workspaces Operationalizing Microservices in the Cloud Introducing GitHub Codespaces for Streamlined Development Compiling Python Projects in GitHub Codespaces Mastering Continuous Integration in GCP Course Wrap-up and Future Steps in Your GCP Journey Learning Objectives Design and optimize cloud solutions Analyze and optimize technical and business processes Manage and provision cloud infrastructure Ensure solution and operations reliability By the end of this course, you will have the knowledge and skills needed to pass the Google Cloud Certified Professional Cloud Architect exam and to effectively design and manage cloud solutions using Google Cloud Platform. Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: Professional Data Engineer Certification Course Section 1: Designing Data Processing Systems 1.1 Storage technology selection 1.2 Data pipeline design 1.3 Data processing solution design 1.4 Data warehousing & processing migration Section 2: Building & Operationalizing Data Processing Systems 2.1 Storage system implementation 2.2 Pipeline building & operationalization 2.3 Processing infrastructure implementation Section 3: Operationalizing ML Models 3.1 Pre-built ML models as a service 3.2 ML pipeline deployment 3.3 Training & serving infrastructure selection 3.4 ML model measurement, monitoring & troubleshooting Section 4: Ensuring Solution Quality 4.1 Security & compliance design 4.2 Scalability & efficiency assurance 4.3 Reliability & fidelity assurance 4.4 Flexibility & portability assurance Learning Objectives Learn to design, build, and operationalize data processing systems that meet business requirements, system requirements, and industry best practices. Learning to build scalable, efficient, and secure solutions with Google Cloud technologies to manage and analyze data at scale. Learning the key syntax and features of the SQL language for handling, querying, and performing operations on structured data in Google Cloud. Additional Popular Resources Pytest Master Class AWS Solutions Architect Professional Course Github Actions and GitOps in One Hour Video Course Jenkins CI/CD and Github in One Hour Video Course AWS Certified Cloud Practitioner Video Course Advanced Testing with Pytest Video Course AWS Solutions Architect Certification In ONE HOUR Python for DevOps Master Class 2022: CI/CD, Github Actions, Containers, and Microservices MLOPs Foundations: Chapter 2 Walkthrough of Practical MLOps Learn Docker containers in One Hour Video Course Introduction to MLOps Walkthrough AZ-900 (Azure Fundamentals) Quick reference guide 52 Weeks of AWS Episode 8: Infrastructure as Code with CDK and AWS Lambda Learn GCP Cloud Functions in One Hour Video Course Python Devops in TWO HOURS! MLOps Platforms From Zero: DatMLOps Platforms From Zero: Databricks, MLFlow/MLRun/SKLearn AWS Machine Learning Certification In ONE HOUR Fast, documented Machine Learning APIs with FastAPI Zero to One: AWS Lambda with SAM and Python in One Hour AWS Storage Solutions 2022: EBS/S3/EFS/Glacier Python Bootcamp for Data Testing In Python book Minimal Python book Practical MLOps book Python for DevOps-Playlist.

Abstract: DevOps command-line tools in Python and Rust Use both Python and Rust to automate systems engineering tasks In this introductory course, you will learn how to build powerful command-line tools using Python and Rust. You will discover how to set up a development environment, incorporate user input, expand functionality using modules and libraries, and optimize performance for common DevOps and Systems Engineering tasks. Whether you're new to programming or an experienced developer, this course will teach you the fundamentals of building command-line tools using Python and Rust. By the end of the course, you'll have the skills to automate tasks and improve the efficiency of your DevOps and Systems Engineering workflows. This course includes GitHub repositories that you can use as a reference for all the learning content: Python CLI Examples Rust CLI Examples Learn objectives Set up a development environment for building command-line tools. Build basic command-line tools. Expand the functionality of command-line tools using modules and libraries. Create a simple command-line tool using Rust and understand the basic structure of a Rust-based CLI tool. Incorporate user input, such as arguments and options. Learn how to organize code into modules and packages, and work with dependencies and libraries in both Python and Rust. About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his past experience as a Linux system administrator is seen throughout this course, where you will get a first-hand experience with high-level knowledge and practical examples. Resources Pytest Master Class Practical MLOps book.

Abstract: Introduction to MLflow for MLOps Learn how to use MLflow for managing the machine learning lifecycle. Track experiments, package models, and deploy to production. In this course you'll learn how to use MLflow - an open source platform for managing the machine learning lifecycle. You'll learn how to: Install MLflow and explore its components like the UI, tracking, and model packaging Log metrics, parameters, and artifacts to track ML experiments Create reproducible ML projects with MLflow for repeatable model training Package models and dependencies for deployment and serving Use model registries to version, stage, and deploy models Deploy models to tools like Azure ML and SageMaker This course includes hands-on exercises, projects, and real-world examples so you can apply your new MLflow skills immediately. Use the reference repository for MLFlow examples and projects: Example MLFlow Projects Learning objectives Install and configure MLflow Use the tracking UI and APIs Log metrics, parameters, tags, and artifacts Create reproducible ML projects Version, stage, and deploy models with registries Deploy models to Azure ML, SageMaker, etc Lesson 1: Introduction to MLflow Lesson Outline Overview of MLflow components Installation and configuration Tracking experiments with UI, Python, R APIs Logging metrics, params, tags, artifacts Lesson 2: MLflow Projects Lesson Outline Motivation for reproducible ML projects Creating project directories Running projects locally or on Git Customizing execution environments Lesson 3: MLflow Models Lesson Outline Packaging models and dependencies Model versioning with registries Staging and promoting model stages Deploying models to services About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University teaching Machine Learning, Cloud Computing, Data Engineering, Python, and Rust. With Alfredo's guidance, you will gain the knowledge and skills to work with MLFlow and apply it to MLOps tasks. Resources Pytest Master Class Practical MLOps book.

Abstract: Scripting with Python and SQL for Data Engineering Learn Python data structures, web scraping, SQL, and MySQL from the ground up. Master essential skills for collecting, storing, and working with data. In this hands-on course for beginners, you'll learn how to: Store and manipulate data using Python lists, dictionaries, JSON Write reusable scripts to process data Connect Python to databases like SQLite and MySQL Query, import, and export data with SQL Scrape and parse websites using Beautiful Soup and Scrapy Persist scraped data to databases and files You'll use the following example repositories to practice: Mapping data in Python Python Scripting Scrapping Basics Key topics include: Mapping, iterating, and persisting data structures Creating modules, scripts, and workflows in Python SQL essentials - queries, statements, aggregations Setting up connections from Python to SQLite and MySQL Scraping data locally and at scale with spiders Storing scraped data to optimize pipelines You'll build your data wrangling skills through practical examples and hands-on coding exercises in every lesson. By the end of the course, you'll have experience building end-to-end data engineering scripts. Whether you're a beginner looking to learn Python and SQL, or want to develop robust data engineering skills, this course will get you started. Enroll now and start collecting, storing, and working with data using Python and SQL You'll gain hands-on experience building Python scripts and SQL queries for common data engineering tasks. This course is divided in 4 weeks: Week 1 Working with Data in Python By the end of Week 1 you'll be able to: Apply Python data structures like lists, dicts Extract data from sources like CSV, JSON Load and persist data using JSON Lesson 1: Data Structures in Python Lesson Outline Lists, tuples, dictionaries Working with pandas DataFrames Loading data files like CSV into data structures Lesson 2: Reading and Writing Data Lesson Outline Reading and writing CSV files Serializing Python objects with JSON Parsing and dumping JSON data Lesson 3: Persisting and Loading Data in Python Lesson Outline Loading data from files Saving data from Python to disk Loading and saving data to JSON Week 2 Python Scripting and SQL By the end of Week 2 you'll be able to: Write reusable Python scripts Use SQLite to persist data Query SQLite databases with Python Lesson 1: Python Scripting Techniques Lesson Outline Writing modular, reusable Python scripts Exception handling and logging Python virtual environments Lesson 2: Python with SQLite Lesson Outline Creating SQLite databases from Python Writing tables with SQLAlchemy Querying SQLite from Python with SQLAlchemy Week 3 Learning Objectives By the end of Week 3 you'll be able to: Scrape and collect data from websites Build scalable scraping scripts Persist scraped data to files/databases Lesson 1: Web Scraping with Python Lesson Outline HTML parsing and structure Using Beautiful Soup for scraping Storing scraped data in Python Lesson 2: Scalable Web Scraping Lesson Outline Scraping best practices Scaling scraping with multiprocessing Storing scraped data in databases Week 4 Learning Objectives By the end of Week 4 you'll be able to: Connect to MySQL from Python Execute SQL statements and queries Import and export data from MySQL Lesson 1: Python and MySQL Lesson Outline Installing MySQL and configuration Connecting Python to MySQL Executing queries and statements Lesson 2: Running SQL queries from VSCode Use Visual Studio Code to build SQL queries Execute and review SQL queries from Visual Studio Code Lesson 3: Importing and Exporting Data Lesson Outline Loading and exporting CSV data Best practices for moving data into MySQL Automating data imports with Python About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his experience teaching and authoring content about DevOps and MLOps will give you everything you need to get started applying these powerful concepts. Resources Python and Rust CLI Tools Linux For Beginners Hands on Python for MLOps.

Abstract: Microsoft Azure AI Fundamentals (AI-900) Certification Pass the AI Fundamentals certification This early release (ongoing) course will give you everything you need to pass the Azure AI Fundamentals (AI-900) certification. This is the foundational certification for doing Machine Learning and Data Science using Azure and its specialized services. The course will go into every content that is relevant to pass the certification including how to create a study guide and strategy to pass the exam. This course includes and covers all of the author's study notes so that you can use them in addition to your own notes . Course overview This course contains a complete walkthrough over all of the content relevant for the AI-900 certification with a summary and specialized notes after each sections, following the same content that is available as Learning Paths. Aside from the video content, create your own study notes and strategy using the example reference guide Course Content Get Started with AI Get Started with AI Study notes Get Started with Artificial Intelligence on Azure About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his experience studying and passing the AI-900 is seen throughout this course, where you will get a first-hand experience with study notes, example repositories, and a solid study strategy. Resources Pytest Master Class Practical MLOps book.

Abstract: This introductory lesson will walk you through everything you need to know to quickly get started with Azure ML Studio using AutoML. After creating the necessary resources, you'll walk through the steps needed to create a regression job using a dataset. Next you'll create a compute cluster and review the different options available and finally, you'll analyze the results after the best model is selected.

Abstract: This video series covers live coding with TensorFlow and the Rust language.

Abstract: Linux for Beginners A hands on and practical introduction of Linux This early release (ongoing) course will give you a foundational knowledge to quickly get started with Linux. You will learn practical examples packaged in easy-to-follow videos that give you just enough knowledge to use Linux. There is a strong focus on using the terminal exclusively for this course, which is the best way to ensure a wide variety of Linux environments can be used with almost no difference in concepts. The course will not cover deep Linux internals or advanced usage. It will also not go through specialized scenarios that are meant to pass a Linux certification. Learn objectives Use a terminal to navigate the file system Understand different Linux versions and use readily available versions without installing Linux Use a shell with common features Create and edit files using a terminal text editor Manage system processes and services Install and remove packages About your instructor Alfredo Deza has over a decade of experience as a Software Engineer doing DevOps, automation, and scalable system architecture. Before getting into technology he participated in the 2004 Olympic Games and was the first-ever World Champion in High Jump representing Peru. He currently works in Developer Relations at Microsoft and is an Adjunct Professor at Duke University. This solid background in technology and teaching, including his past experience as a Linux system administrator is seen throughout this course, where you will get a first-hand experience with high-level knowledge and practical examples. Resources Pytest Master Class Practical MLOps book.

Abstract: This table includes statistical projections on the energy balance for IEA countries by analyzing their respective products (coal, oil, nuclear, hydro, biofuels and more) in terms of flow. Data are expressed in thousand tonnes of oil equivalent(ktoe) and forecasts extend to 2030.

Abstract: This dataset includes data on Services energy end use, Services activity, Services emissions as well as Services energy and carbon indicators.

Abstract: The global oil and gas industry encompasses a large and diverse range of players: from small, specialised operators to huge national oil companies. These producers face pivotal choices about their role in the global energy system amid a worsening climate crisis fuelled in large part by their core products. The Oil and Gas Industry in Net Zero Transitions analyses the implications and opportunities for the industry that would arise from stronger international efforts to reach energy and climate targets. It also examines how transitions increase the likelihood of boom and bust cycles for oil and gas producer economies. It highlights strategies for producer economies that could complement broader reforms to build macroeconomic stability and the role of international partners to support this process. The report sets out a fair and feasible way forward in which oil and gas companies and producer economies take a real stake in the clean energy economy while helping the world avoid the most severe impacts of climate change.

Abstract: Establishing appropriate institutional architecture is important to integrate power systems across borders and facilitate electricity trading, as even if the necessary infrastructure is in place, it does not automatically follow that it is being used to exchange power effectively. The co-ordination of all stakeholders – governments, utilities1 and regulators – is required within jurisdictions, as is the creation of regional entities to support and oversee the integration process. This report therefore examines stakeholder roles at different stages of crossborder integration to enable multilateral power trade. As energy regulators are our main audience, we focus on their roles and responsibilities after briefly addressing those of governments and utilities.

Abstract: Government action plays a pivotal role in ensuring secure and sustainable energy transitions and combatting the climate crisis. Energy policy is critical not just for the energy sector but also for meeting environmental, economic and social goals. Governments need to respond to their country’s specific needs, adapt to regional contexts and help address global challenges. In this context, the International Energy Agency (IEA) conducts Energy Policy Reviews to support governments in developing more impactful energy and climate policies. This Energy Policy Review was prepared in partnership between the Government of Estonia and the IEA. It draws on the IEA's extensive knowledge and the inputs of expert peers from IEA member countries to assess Estonia’s most pressing energy sector challenges and provide recommendations on how to address them, backed by international best practices. The report also highlights areas where Estonia’s leadership can serve as an example in promoting secure clean energy transitions. It also promotes the exchange of best practices among countries to foster learning, build consensus and strengthen political will for a sustainable and affordable clean energy future.

Abstract: Tackling methane in the coal sector is a major opportunity for climate action that can also strengthen energy security. Experience shows that there are several steps countries can take today – using existing technologies and tools – that can lead to significant reductions in methane emissions from coal mining. This report highlights the lessons learned in different coal-producing jurisdictions to support the development of smart and effective methane regulation. It then provides detailed guidance on the process of designing, drafting and implementing new regulations. Finally, it discusses the different regulatory approaches currently in use for methane, with the aim of providing a comprehensive toolkit for policy makers.

Abstract: This dataset provides crude oil imports costs by type of crude in terms of country of origin.

Abstract: This dataset provides wholesale and retail energy end-use price indices by sector for both household and industry in nominal and real terms. Products such as coal, electricity, natural gas and oil products are included within the table.

Abstract: This dataset includes various crude oil products listed by their respective market spot prices.

Abstract: This database comprises various energy product prices expressed in national currency per toe with products such as sulphur fuel oil, automotive diesel, premium leaded gasoline while comparing data by sectors.

Abstract: This dataset includes energy statistics on the basis of main economic indicators such as Consumer Price Index, Purchasing Power Parities, Producer Price Index and currency exchange rates.

Abstract: This dataset provides wholesale and retail energy end-use price indices by sector for both household and industry in nominal and real terms. Products such as coal, heavy fuel oil, automotive diesel fuel, natural gas, oil products, unleaded gasoline are the core focus of this dataset.

Abstract: This dataset includes oil product spot prices by market. Products such as gasoil, gasoline, high sulphur fuel oi, jet kerosene, naphtha are listed within the database.

Abstract: This database comprises various energy product prices expressed in US dollars with products such as sulphur fuel oil, automotive diesel, premium leaded gasoline while comparing data by sectors.

Abstract: This database comprises various energy product prices expressed in national currency per unit with products such as natural gas, steam coal, coking coal, liquefied petroleum gas while comparing data by sectors.

Abstract: The world remains far off track to reach universal access to electricity by 2030 – a key target canonised in UN Sustainable Development Goal 7 (SDG7). While many countries have set targets to advance universal electrification, it remains challenging to collect timely data on their progress, making it difficult to build or refine their electrification plans and policy strategies. To provide governments with the tools to achieve their goals, the Guidebook for Improved Electricity Access Statistics (hereafter “Guidebook”) focuses on methodologies using readily available supply-side data from electric utilities, mini-grid operators, and off-grid system distributors to track access to electricity trends. This approach can be adopted at a low-cost and provides accurate estimates on access rates with a potential time lag on the order of months, instead of years. Supply-side data complements household or census surveys, which can give a more nuanced, detailed picture, but are typically only run every five to ten years due to their expense. This report provides step-by-step guidelines on how to implement an improved supply-side data collection process, and produce more time-sensitive and comprehensive access to electricity indicators. In addition, the Guidebook proposes a standard method to capture mini-grids and stand-alone off-grid systems in calculating access to electricity, which are becoming increasingly more commonplace in strategies to reach universal access. Finally, the Guidebook suggests further steps practitioners can take to advance geospatial data collection and reporting, which has become the new best practice for electrification planning supporting both electric utilities and the vibrant off-grid industry.

Abstract: Renewables are growing rapidly in the electricity systems around the world as countries seek to improve their energy security, meet emission reduction targets and take advantage of cheaper electricity sources. Thanks to successful use of flexibility resources – from stronger grids and interconnections to demand-side measures, affordable storage and dispatchable power supply – many countries have already securely and efficiently integrated significant shares of variable renewables (VRE) in their electricity generation. As wind and solar continue to grow as a proportion of generation, system level surpluses and periods of lower generation will eventually expand beyond hour-to-hour or daily variations to seasonal timescales. Addressing seasonal variability of renewables means that flexibility resources will be needed to varying extents throughout the year, even on a week-to-week or month-to-month basis.

Abstract: Today, oil and gas operations account for around 15% of total energy-related emissions globally, the equivalent of 5.1 billion tonnes of greenhouse gas emissions. In the International Energy Agency’s Net Zero Emissions by 2050 Scenario, the emissions intensity of these activities falls by 50% by the end of the decade. Combined with the reductions in oil and gas consumption in this scenario, this results in a 60% reduction in emissions from oil and gas operations to 2030. Fortunately, oil and gas producers have a clear opportunity to address the problem of emissions from their activities through a series of ready-to-implement and cost-effective measures. These include tackling methane emissions, eliminating all non-emergency flaring, electrifying upstream facilities with low-emissions electricity, equipping oil and gas processes with carbon capture, utilisation and storage technologies, and expanding the use of hydrogen from low-emissions electrolysis in refineries. Upfront investments totalling USD 600 billion would be required to halve the emissions intensity of oil and gas operations globally by 2030. This is only a fraction of the record windfall income that oil and gas producers accrued in 2022 – a year of soaring energy prices amid a global energy crisis. This report aims to inform discussions on these issues in the run-up to the COP28 Climate Change Conference in Dubai in November and is part of a broader World Energy Outlook special report to be released later in 2023 focusing on the role of the oil and gas industry in net zero transitions.

Abstract: The implementation phase for achieving a net zero steel industry will require robust methodologies for measuring emissions at the site- and product-level, together with data collection frameworks to facilitate comparison and track progress. An existing array of methodologies and frameworks for the steel industry provide a good starting point for efforts to achieve these outcomes, but much work remains to achieve interoperability, transparency and fitness for purpose for net zero. Following an evaluation of these existing methodologies and frameworks, this report provides “net zero principles” to guide potential next steps for their development and implementation, together with specific recommendations for G7 members. In the context of Japan’s G7 Presidency, the Ministry of Economy, Trade and Industry requested the International Energy Agency (IEA) to examine the topic of Emissions Measurement and Data Collection for a Net Zero Steel Industry. This work complements that undertaken during Germany’s G7 Presidency in 2022 – Achieving Net Zero Heavy Industry Sectors in G7 Members – providing insights and direction for the G7 Industrial Decarbonisation Agenda with regard to tackling industrial emissions.

Abstract: Finland has set one of the most ambitious climate targets in the world, a legal obligation to reach carbon neutrality by 2035. It has made notable progress towards this target, deploying the first new nuclear reactor in Europe in over 15 years and strongly expanding wind generation. Thanks to the progress Finland has made on its clean energy transition, the country has the second lowest share of fossil fuels in its energy supply among IEA members. It is also reducing its reliance on Russian energy imports and ensuring energy security by increasing imports from other countries, raising domestic renewable energy production and improving energy efficiency. Despite these notable successes in clean energy and energy security, significant challenges remain. Imported fossil fuels still account for over a third of the energy supply while some areas of the Finnish economy, such as transport and key industrial activities, remain dependant on fossil fuels. Also, land use change and forestry in Finland, which have historically offset a significant amount of greenhouse gas emissions, became a net source of emissions for the first time in 2021. In this report, the IEA provides a range of energy policy recommendations to help Finland smoothly manage the transition to a secure, efficient and flexible carbon neutral energy system.

Abstract: This report was commissioned by Korea's Ministry of Trade, Industry and Energy and carried out jointly by the International Energy Agency (IEA) and the Korea Energy Economics Institute. The objective of the study was to provide high-level policy recommendations on Korea’s clean energy transition from coal in the power sector. The report covers a detailed review of policy and market developments around Korea’s transition to net zero. The scope of the report includes all sectors of the economy, industry and all the regions across the world, where we extract the main recommendations that are applicable to the case of Korea. Currently, the power sector is the largest CO2-emitting sector and coal is the single biggest source of CO2 emissions, as it is the backbone of many electricity systems. Thus, coal power plants have been a target for reaching net zero emissions by 2050 for long time. Korea has firm objectives to achieve carbon neutrality by 2050, aiming to accelerate the clean energy transition of coal power plants. Policy recommendations were formulated around two priorities: affordable and secure supply of electricity and people-centred transition.

Abstract: Since the last review in 2016, Italy has raised its climate ambitions by aiming for carbon neutrality by 2050, and the country is on track to reach its 2030 targets for emissions reductions and energy efficiency. The government has taken encouraging initial steps to overcome the long permitting procedures, administrative burdens and increasing local opposition that have delayed new renewable installations. Italy in 2022 successfully reduced its reliance on Russian natural gas imports, by signing new contracts with alternative suppliers, making use of the pipeline and LNG infrastructure that it has built up over the last decade. Reducing overall demand for natural gas through an accelerated shift to alternative energy sources and a stronger focus on energy efficiency, especially in the building sector, will not only further strengthen energy security, but also help the country meet its climate targets. In this report, the IEA provides energy policy recommendations to help Italy effectively transform its energy sector in line with its goals.

Abstract: The International Energy Agency (IEA) and the Institute of Energy Economic, Japan (IEEJ) have developed and published long-term decarbonisation pathways for Southeast Asia and Indonesia. This report provides a comparison of modelling approaches, quantitative drivers, and results from the IEA and IEEJ pathways, highlighting areas of agreement, as well as identifying and explaining differences, and thereby to derive implications.

Abstract: The Clean Energy Transitions Programme 2022 (CETP) provides an overview of the programme’s major achievements in the last year. During this time, the CETP has driven forward an ambitious and impactful agenda, even as the world has contended with multiple overlapping crises. As just some examples, the CETP delivered the Energy Sector Roadmap to Net Zero Emissions in Indonesia , which ministers strongly welcomed at the G20, and has played a key role in shaping Indonesia’s landmark Just Energy Transition Partnership (JETP). Similarly, the IEA has been able to take its engagement to a whole new level with various African countries, such as with its first-ever report on the Clean Energy Transitions in the Greater Horn of Africa , and provided policy advice that was directly accepted into China’s latest five-year plan. More broadly, it has also supported strategically important work on critical minerals, energy employment and clean energy investment. Overall, the CETP’s combination of data, insight, advice and capacity building is making a strong positive contribution to advancing clean energy transitions around the world – with a focus on emerging and developing economies. In the five years since its launch, the CETP has become the largest and most important programme at theInternational Energy Agency (IEA). It has been the main vehicle through which theIEA has established, deepened and strengthened partnerships with major nonmember countries – those countries that are at the forefront of the clean energytransition. The programme has also played a crucial role in enabling the IEA tostrengthen its global leadership role in clean energy transitions at a pace and scalethat would otherwise not have been possible, with great benefit to theprogramme’s focus countries and IEA member countries alike.

Abstract: Power markets are an instrument used globally to ensure electricity security while maintaining affordability and incentivising decarbonisation. The People’s Republic of China (hereafter, “China”) has been making big steps towards implementing markets, and the goals announced in 2020 of carbon dioxide emissions peaking before 2030 and carbon neutrality before 2060 have added momentum to expand their footprint. Provinces have taken a leading role in designing and implementing markets. To improve sharing of resources, the guidance to implement a unified national market system pushes for more co-ordination between provinces. This report examines the role of power markets in China and the pathways to develop a national market. The analysis focuses on short-term markets because they have the potential to unlock flexibility the system needs in light of renewables growth and changing weather patterns. The report provides recommendations to improve markets’ co-ordination across the country as well as within the provinces.

Abstract: New Zealand has set ambitious targets for reducing greenhouse gas emissions, including achieving net zero emissions by 2050. The country enjoys many natural advantages for its energy transition, including an enviable renewable resource base. New Zealand already has a low‑emissions electricity system, with significant production from both hydropower and geothermal, and therefore has an attractive opportunity to leverage this clean electricity to decarbonise end-user sectors. This will require not only sizeable technological investments to efficiently electrify transport and industry, but will also necessitate a sizeable buildout of additional renewables generation capacity, along with supplemental grid and storage investments. Notably, the transport sector accounts for the highest share of emissions and is almost entirely dependent on oil. Industry is also a major contributor to New Zealand’s emissions and is heavily reliant on fossil fuels. In this report, the IEA provides energy policy recommendations to help New Zealand effectively manage the transformation of its energy sector in line with its climate targets.

Abstract: Towards Hydrogen Definitions Based on their Emissions Intensity is a new report by the International Energy Agency, designed to inform policy makers, hydrogen producers, investors and the research community in advance of the G7 Climate, Energy and Environmental Ministerial meeting in April 2023. This report assesses the greenhouse gas emissions intensity of the different hydrogen production routes and reviews ways to use the emissions intensity of hydrogen production in the development of regulation and certification schemes. An internationally agreed emissions accounting framework is a way to move away from the use of terminologies based on colours or other terms that have proved impractical for the contracts that underpin investment. The adoption of such a framework can bring much-needed transparency, as well as facilitating interoperability and limiting market fragmentation, thus becoming a useful enabler of investments for the development of international hydrogen supply chains.

Abstract: Greece aims to reduce total greenhouse gas emissions by 55% by 2030 and reach net zero emissions by 2050. It has made notable progress towards meeting these targets. This includes significantly reducing its reliance on lignite and setting a binding target to end lignite-fired power generation by 2028 while working to ensure a just transition in its lignite mining regions and reducing energy poverty. Greece has also made strong progress on renewable energy, which covered 20% of its total final energy consumption in 2021. Following Russia’s invasion of Ukraine, Greece is reducing its reliance on Russian energy imports and ensuring secure access to energy by diversifying its supply and increasing domestic energy production. Despite these notable successes, significant challenges remain. Fossil fuels still account for most of Greece’s energy use and stronger efforts are needed on energy efficiency. n this report, the IEA provides a range of energy policy recommendations to help Greece smoothly manage its transition to a secure, efficient and flexible carbon neutral energy system.

Abstract: Countries around the world are facing an energy security and affordability crisis affecting households, businesses and public budgets. In Latin America, household expenditure on energy services is second only to food. On average, the share of household budget for energy ranges between 7% and 9% of income – and up to more than 24% for the poorest households. As governments urgently implement measures to protect citizens and secure their economies, it is important to consider short- and long- term solutions to ease existing pain and reduce risks over time. While much attention focuses on securing energy supplies, it is equally important to take a hard look at the role of more efficient and flexible demand in securing affordable, reliable energy.Decades of policies and programmes have delivered energy savings around the world. As a result, energy systems are more resilient, bills are lower, and public budgets feel less pressure than they would have without these efforts. This report looks at energy efficiency and demand management policies and programmes across Latin America. It presents evidence of the results of these efforts on improving living standards, public budgets, energy security and employment, and provides a view on the opportunities to expand these benefits and support just energy transitions.

Abstract: Energy efficiency indicators are key to tracking energy efficiency progress for a variety of purposes (e.g. policy making, monitoring targets, making energy projections, developing scenarios and planning, and benchmarking). This guide is for professionals and decision makers, describing options and good practices for the collection of energy end-use data and the development of energy efficiency indicators at the national level. In parallel, it can also be used as an assessment tool, helping countries/economies to locate their starting point, and to identify appropriate targets according to their respective national interests and priorities.The roadmap presented here encompasses the results of a consultation exercise across countries and presents good practices and practical tips. It acknowledges that there is no single solution, but a number of possible pathways instead, depending on national contexts and priorities. The roadmap is a strategic document looking at the whole value chain in the development of efficiency indicators, from the initial point where the need for data and indicators arises up to the later dissemination and data use stages, and is meant to be a useful resource for practitioners across the globe in the development of energy efficiency indicators.

Abstract: This dataset on energy prices provides statistics on the crude oil import costs both in terms of USD/bbl or real index per national currency.

Abstract: This dataset provides statistics on electricity imports by country of origin.

Abstract: This table presents detailed coal export data by destination for OECD member states for the following coal products and aggregates: anthracite, coking coal, sub-bituminous coal, lignite/brown coal, coke oven, coke, hard coal, brown coal, steam coal, other bituminous coal and anthracite. The data are reported in kilotonnes.

Abstract: This table focuses on the export of various energy products such as crude oil, refinery gas, natural gas liquids, fuel oil amongst OECD member countries.

Abstract: This dataset provides statistics on electricity supply per OECD country with net electricity production from main sources and trade data from January 2000 onwards. Data are presented on a monthly basis.

Abstract: This table presents gross electricity and heat production from each energy source such as nuclear, hydro, combustible fuels, geothermal, solar, wind and other sources as well as the breakdown according to the type of plant (electricity only, CHP or heat only). Inputs by type of fuel are also provided. Gross heat production is the total heat produced by the installation and includes the heat used by the installation's auxiliaries which use a hot fluid (space heating, liquid fuel heating, etc.) and losses in the installation/network heat exchanges whereas Net Heat Production is the heat supplied to the distribution system as determined from measurements of the outgoing and return flows.

Abstract: Since the publication of its latest Power Development Plan (PDP) in 2020 (PDP 2018 Revision 1), Thailand has considerably increased its emissions reductions objectives, announcing a net zero greenhouse gas emissions target for 2065 and carbon neutrality for 2050. As the power sector is a large part of the country's emissions, and because it has a key role to play in decarbonising other sectors, meeting these targets is possible only if the power sector is decarbonising too. This report hence analyses how Thailand can achieve its clean electricity transition, by comparing the planned trajectory of the PDP with the emissions targets, and providing an assessment of the gaps. Building upon the current PDP, this report analyses how the Thai power system can decrease its emissions to meet the targets by increasing the amount of wind and solar PV in its system, and how it can integrate these variable renewable energy sources efficiently. This report concludes work area one of the joint work programme among the Electricity Generating Authority of Thailand (EGAT), the Ministry of Energy of Thailand and the International Energy Agency (IEA), and has benefited from data and input from the Thai counterparts. The analysis is based on a PLEXOS model of the Thai power system that has been developed by the IEA in cooperation with EGAT.

Abstract: A future with net zero emissions requires scaling up improvements in energy efficiency, electrification of end uses and renewable energy generation. For the Association of Southeast Asian Nations (ASEAN), a growing population and rising standards of living will massively increase future energy demand. Mitigating growing electricity demand and integrating renewable energy into electricity generation will therefore be paramount for the region's clean energy transition and avoid lock-in of additional fossil fuel generation. The intermittent nature of variable renewable energy and increasing deployment of distributed energy resources are putting additional pressure on existing grids. In response to these challenges, this report explores the opportunities and challenges for efficient grid-interactive buildings in the ASEAN region. Such buildings are becoming a crucial element for the global ambition to attain net zero emissions, as they can combine enhanced energy efficiency, advanced smart digital technologies and decarbonised electricity generation. This integration creates the potential for buildings to shift from energy-intensive consumers to low-carbon prosumers, empowered by digital technologies that can offer flexibility benefits to the electricity grids. With the ability to produce, consume, store, sell and buy energy, buildings become active participants in the building-to-grid ecosystem. Drawing on relevant international trends and best practices, the current report lays out an analytical framework to assess a variety of factors that can enable a building to become energy efficient and grid-interactive. The framework is used to analyse the current situation in countries of the ASEAN region. The report provides policy-oriented recommendations and guidelines tailored to different stages of the process for adopting efficient grid-interactive solutions in buildings. These recommendations can support ASEAN policy makers in their policy development to create an energy-efficient and grid-interactive built environment, contributing to a cleaner and more sustainable energy future.

Abstract: Technology manufacturing plays a pivotal role in the energy transition required to meet climate, energy security and economic development goals. Deploying clean energy technologies at the pace required to put the world on a trajectory consistent with net zero emissions by mid-century will demand rapid expansion in manufacturing capacity, underpinned by secure, resilient and sustainable supply chains for their components and materials. The State of Clean Technology Manufacturing: Energy Technology Perspectives Special Briefing provides an update on recent progress in clean energy technology manufacturing in key regions. It focuses on five technologies – solar PV, wind, batteries, electrolysers and heat pumps – that will be critical to the energy transition. Manufacturing capacity for these technologies is expanding rapidly, driven by supportive policies, ambitious corporate strategies and consumer demand. The aim is to keep decision makers informed of investment trends and the impact that recent industrial strategies are having in these highly dynamic sectors. This special briefing was produced to support deliberations at the 2023 G7 Leaders' Summit in Hiroshima, Japan, from 19-21 May 2023. It builds on analysis in the latest edition of the IEA's flagship technology publication, Energy Technology Perspectives 2023 (ETP-2023), published in January 2023, to take into account the latest announced expansions in manufacturing capacity.

Abstract: The oil and gas industry has some of the best and most cost-effective opportunities to reduce methane emissions. The potential to do so is clear. Some countries and companies have already demonstrated that achieving near-zero emissions from oil and gas operations is technically and economically possible. There are a growing number of initiatives, policies and regulations aiming to reduce emissions globally, and many reductions can be realised while saving money. However, overall progress has been much too slow, despite the record profits that the oil and gas industry saw in 2022. This report looks in detail at the investment requirements to deliver a sharp reduction in oil and gas methane emissions to 2030, and how these could be financed. The analysis is intended to inform discussions in the runup to COP28 and help prompt the necessary actions to accelerate the pace of change.

Abstract: The clean energy transition requires a fundamental transformation of power systems, including much higher levels of digitalisation at scale across all grid domains, from generation to transmission and distribution to end-use. Strong policy attention is required to scale up investments in smarter and more resilient grids in emerging and developing economies where electricity consumption is set to grow at a rapid rate while also providing greater levels of electricity access. Investments in smarter and more resilient grids will be necessary to accommodate the greater deployment of renewable energy and enhance energy security. Digital technologies designed for power systems are instrumental to unlock essential system services required to integrate high shares of variable renewable energy. They can also provide solutions to leverage data flows, connectivity, and management across the whole electricity system. To unlock these digital opportunities, adequate planning, investment, and policy action are needed. As part of the Digital Demand Driven Electricity Networks (3DEN) initiative, this report provides guidance for energy policy makers on possible ways to enable and drive investments in smart and resilient electricity grids. It also gives suggestions on how to start creating an environment that supports the effective use of innovative digital technologies within the electricity sector. It draws on examples and case studies to show the wide range of digital opportunities and solutions that can help governments implement efficient and smart power systems.

Abstract: In this table, statistics for different types of coal and coal products, including manufactured gases is shown.This table provides full supply and consumption data of coal such as primary energy supply,transformation sector, energy sector and final consumption based in kilotonnes for OECD countries and OECD country aggregate groups.

Abstract: This table on air and climate contains indicators for carbon dioxide (CO2) emissions generated from fossil fuel combustion in IEA countries and regions. Data are available from 1960 onwards.

Abstract: This dataset comprises statistics regarding various renewable and waste energy while presenting trade statistics of various renewable energy products as well as figures for those products in terms of electricity output (GWh) and heat output(TJ).

Abstract: In this table, calorific values for different types of coal and coal products are shown on a country by country basis. The aggregates for hard coal and brown coal are included for years prior to 1978 only. This table provides the calorific values used to convert raw tonnes of coal and coal products into energy for the OECD Coal Balance data. It is shown in megajoules per tonne for OECD countries.

Abstract: This table contains data on natural gas exports by origin for OECD countries.

Abstract: This dataset includes statistics on energy technology Research & Development and dissemination as well as R&D budget for IEA countries. It presents shifts in R&D, expenditures associated with investments and further analyzes budget allocations in terms of flow.

Abstract: This table contains world data on supply of natural gas.

Abstract: This table provides a reduced electricity and heat balance (145 countries + 22 aggregates; 8 flows).

Abstract: This table includes monthly OECD net electricity production from main sources and trade data from January 2000 to December 2010.

Abstract: This dataset provides data on energy world indicators in terms of world supply, consumption and trade.

Abstract: This table includes the average conversion factors from tonnes to barrels, for the OECD countries and for 24 products.

Abstract: This dataset brings together statistics compiled by the IEA on natural gas imports by origin. It contains time series for 1993 onwards for the world, 143 countries and 24 regional aggregates on imports of natural gas by country of origin.

Abstract: This dataset provides statistics on electricity exports by destination country.

Abstract: This dataset covers 30 countries and 1 indicator.

Abstract: This dataset comprises a detailed set of statistics on carbon dioxide (CO2) emissions estimates from fossil fuel combustion. It includes an extensive set of products emitting CO2 from combustion analyzed in terms of their respective industry flows (iron and steel, manufacturing industries, agriculture, machinery, etc). Data are presented from 1960 onwards.

Abstract: This table contains data on energy supply, trade and consumption in original units (1000 metric tonnes, terajoules and gigawatt hours) for coal, oil, gas, electricity, heat, combustible renewables, biogasoline, energy, refinery gas, charcoal, waste.

Abstract: This dataset presents detailed statistics on carbon dioxide (CO2) emissions from fuel combustion by product (coal/peat, oil, gas) and flow (manufacturing, main activity producer, construction and more) according to the 2006 IPCC Guidelines. Data are presented from 1960 onwards.

Abstract: This table presents detailed coal import data by origin for OECD member states for the following coal products and aggregates: anthracite, coking coal, subbituminous coal, ignite/brown coal, coke, oven coke, hard coal, brown coal, steam coal, other bituminous coal and anthracite. The data are reported in kilotonnes.

Abstract: This table contains data on the energy trade, supply and consumption of coal, oil, gas, electricity, heat, combustible renewables and waste, expressed in thousand tonnes of oil equivalent (ktoe).