Merge pull request #52972 from MicrosoftDocs/NEW-DP-databricks-engineering

Databricks release -> main

Author: JamesJBarnett

learn-pr/achievements/azure-databricks.svg

@@ -0,0 +1,40 @@
+### YamlMime:LearningPath
+uid: learn.wwl.azure-databricks-data-engineer-deploy-maintain-data-pipelines-workloads
+metadata:
+  title: Deploy and Maintain Data Pipelines and Workloads With Azure Databricks
+  description: Master the complete lifecycle of building, deploying, and maintaining production-ready data pipelines in Azure Databricks—from design and orchestration to monitoring and optimization.
+  ms.date: 12/09/2025
+  author: weslbo
+  ms.author: wedebols
+  ms.topic: learning-path
+title: Deploy and maintain data pipelines and workloads with Azure Databricks
+prerequisites: |
+  - Good understanding of Azure Databricks workspaces
+  - Familiarity with data engineering concepts and SQL
+  - Experience with Python programming and notebooks
+  - Knowledge of Git version control fundamentals
+summary: |
+  Master the complete lifecycle of building, deploying, and maintaining production-ready data pipelines in Azure Databricks—from design and orchestration to monitoring and optimization.
+
+  By the end of this learning path, you'll be able to:
+  
+  - Design and implement robust data pipelines using notebooks and Lakeflow Declarative Pipelines
+  - Create and orchestrate Lakeflow Jobs with triggers, schedules, and error handling
+  - Apply version control and deploy pipelines across environments using Git and Databricks Asset Bundles
+  - Monitor, troubleshoot, and optimize data workloads for reliability and performance
+iconUrl: /training/achievements/generic-trophy.svg
+levels:
+- intermediate
+roles:
+- data-engineer
+products:
+- azure-databricks
+subjects:
+- data-engineering
+modules:
+- learn.wwl.design-implement-data-pipelines
+- learn.wwl.implement-lakeflow-jobs
+- learn.wwl.implement-development-lifecycle-processes-azure-databricks
+- learn.wwl.monitor-troubleshoot-optimize-workloads-azure-databricks
+trophy:
+  uid: learn.wwl.azure-databricks-data-engineer-deploy-maintain-data-pipelines-workloads.trophy

learn-pr/paths/azure-databricks-data-engineer-deploy-maintain-data-pipelines-workloads/index.yml

@@ -0,0 +1,34 @@
+### YamlMime:LearningPath
+uid: learn.wwl.azure-databricks-data-engineer-prepare-process-data
+metadata:
+  title: Prepare and Process Data with Azure Databricks
+  description: Master the essential skills to build robust, scalable data engineering solutions with Azure Databricks and Unity Catalog. Learn to design effective data models, ingest data from diverse sources, transform raw data into analytics-ready formats, and ensure data quality across your lakehouse architecture.
+  ms.date: 12/09/2025
+  author: weslbo
+  ms.author: wedebols
+  ms.topic: learning-path
+title: Prepare and process data with Azure Databricks
+prerequisites: |
+  - Good understanding of Azure Databricks workspaces and Unity Catalog concepts
+  - Familiarity with SQL and Python programming
+  - Knowledge of fundamental data engineering and data warehouse concepts
+summary: |
+  Master the essential skills to build robust, scalable data engineering solutions with Azure Databricks and Unity Catalog. Learn to design effective data models, ingest data from diverse sources, transform raw data into analytics-ready formats, and ensure data quality across your lakehouse architecture.
+  
+  In this learning path, you'll learn how to build a data engineering workflow using Azure Databricks and Unity Catalog. Starting with foundational data modeling concepts, you'll design schemas and partitioning strategies optimized for analytical workloads. You'll then explore multiple ingestion patterns—from managed connectors to streaming pipelines—to bring data into your lakehouse. Next, you'll apply transformation techniques to cleanse and reshape data for business use. Finally, you'll implement quality controls to maintain data integrity throughout your pipelines. By the end, you'll have the practical skills to architect and build production-ready data solutions in Unity Catalog.
+iconUrl: /training/achievements/generic-trophy.svg
+levels:
+- intermediate
+roles:
+- data-engineer
+products:
+- azure-databricks
+subjects:
+- data-engineering
+modules:
+- learn.wwl.design-implement-data-modeling-unity-catalog
+- learn.wwl.ingest-data-into-unity-catalog
+- learn.wwl.cleanse-transform-load-data-into-unity-catalog
+- learn.wwl.implement-manage-data-quality-constraints-unity-catalog
+trophy:
+  uid: learn.wwl.azure-databricks-data-engineer-prepare-process-data.trophy

learn-pr/paths/azure-databricks-data-engineer-prepare-process-data/index.yml

@@ -0,0 +1,32 @@
+### YamlMime:LearningPath
+uid: learn.wwl.azure-databricks-data-engineer-secure-govern-unity-catalog
+metadata:
+  title: Secure and Govern Unity Catalog Objects in Azure Databricks
+  description: Learn to implement comprehensive security and governance for your data assets in Azure Databricks using Unity Catalog. Master access control strategies, fine-grained permissions, credential management, and governance practices to build a secure and compliant data platform.
+  ms.date: 12/09/2025
+  author: weslbo
+  ms.author: wedebols
+  ms.topic: learning-path
+title: Secure and govern Unity Catalog objects in Azure Databricks
+prerequisites: |
+  - Good understanding of Azure Databricks workspaces and Unity Catalog concepts
+  - Familiarity with SQL and data access patterns
+  - Knowledge of Microsoft Entra ID and Azure security fundamentals
+summary: |
+  Learn to implement comprehensive security and governance for your data assets in Azure Databricks using Unity Catalog. Master access control strategies, fine-grained permissions, credential management, and governance practices to build a secure and compliant data platform.
+
+  In this learning path, you'll learn how to secure and govern your data estate with Unity Catalog. You'll start by implementing robust security controls, including table and schema-level permissions, row filtering, and column masking. You'll discover how to manage credentials securely using Azure Key Vault, service principals, and managed identities. Then, you'll explore governance capabilities such as data lineage tracking, audit logging, and attribute-based access control. Finally, you'll learn to share data securely with external parties using Delta Sharing. By the end, you'll be equipped to build a secure, compliant, and well-governed data platform that meets enterprise requirements.
+iconUrl: /training/achievements/generic-trophy.svg
+levels:
+- intermediate
+roles:
+- data-engineer
+products:
+- azure-databricks
+subjects:
+- data-engineering
+modules:
+- learn.wwl.secure-unity-catalog-objects
+- learn.wwl.govern-unity-catalog-objects
+trophy:
+  uid: learn.wwl.azure-databricks-data-engineer-secure-govern-unity-catalog.trophy

learn-pr/paths/azure-databricks-data-engineer-secure-govern-unity-catalog/index.yml

@@ -0,0 +1,35 @@
+### YamlMime:LearningPath
+uid: learn.wwl.azure-databricks-data-engineer-setup-configure-environment
+metadata:
+  title: Set up and Configure an Azure Databricks Environment
+  description: Build a solid foundation in Azure Databricks by understanding its architecture, integrations, compute options, and data organization capabilities. Learn how Azure Databricks provides a unified platform for data engineering, analytics, and AI workloads in the cloud.
+  ms.date: 12/09/2025
+  author: weslbo
+  ms.author: wedebols
+  ms.topic: learning-path
+title: Set up and configure an Azure Databricks environment
+prerequisites: |
+  - Fundamental knowledge of data analytics concepts
+  - Basic understanding of cloud storage concepts
+  - Familiarity with SQL and data organization principles
+summary: |
+  Build a solid foundation in Azure Databricks by understanding its architecture, integrations, compute options, and data organization capabilities. Learn how Azure Databricks provides a unified platform for data engineering, analytics, and AI workloads in the cloud.
+  
+  In this learning path, you'll explore the fundamentals of Azure Databricks and how it fits into the modern data platform ecosystem. You'll start by provisioning workspaces and understanding core workloads, then dive into the architectural concepts that separate control and compute planes. You'll discover how Azure Databricks integrates seamlessly with Microsoft Fabric, Power BI, Visual Studio Code, and other Microsoft services to create comprehensive solutions. You'll learn to select and configure the right compute resources for your workloads, optimizing for both performance and cost. Finally, you'll master Unity Catalog's organizational structure to effectively manage your data assets. By the end, you'll have the foundational knowledge needed to build scalable data solutions on Azure Databricks.
+iconUrl: /training/achievements/generic-trophy.svg
+levels:
+- intermediate
+roles:
+- data-engineer
+products:
+- azure-databricks
+subjects:
+- data-engineering
+modules:
+- learn.wwl.explore-azure-databricks
+- learn.wwl.understand-azure-databricks-architecture
+- learn.wwl.understand-azure-databricks-integrations
+- learn.wwl.select-and-configure-compute
+- learn.wwl.create-and-organize-objects-in-unity-catalog
+trophy:
+  uid: learn.wwl.azure-databricks-data-engineer-setup-configure-environment.trophy

learn-pr/paths/azure-databricks-data-engineer-set-up-configure-environment/index.yml

@@ -3,7 +3,7 @@ uid: learn.wwl.explore-azure-databricks
 metadata:
   title: Explore Azure Databricks
   description: Explore Azure Databricks
-  ms.date: 09/12/2025
+  ms.date: 12/09/2025
   author: weslbo
   ms.author: wedebols
   ms.topic: module
@@ -18,7 +18,7 @@ abstract: |
   - Describe key concepts of an Azure Databricks solution
 prerequisites: |
   Before starting this module, you should have a fundamental knowledge of data analytics concepts. Consider completing [Azure Data Fundamentals certification](/credentials/certifications/azure-data-fundamentals?azure-portal=true) before starting this module.
-iconUrl: /learn/achievements/describe-azure-databricks.svg
+iconUrl: /learn/achievements/azure-databricks.svg
 levels:
 - intermediate
 roles:

learn-pr/wwl-data-ai/explore-azure-databricks/index.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.cleanse-transform-load-data-into-unity-catalog.introduction
+title: Introduction
+metadata:
+  title: Introduction
+  description: "Introduction to cleansing, transforming, and loading data into Unity Catalog"
+  ms.date: 12/07/2025
+  author: weslbo
+  ms.author: wedebols
+  ms.topic: unit
+  ai-usage: ai-generated
+durationInMinutes: 2
+content: |
+  [!include[](includes/1-introduction.md)]

learn-pr/wwl-databricks/cleanse-transform-load-data-into-unity-catalog/1-introduction.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.cleanse-transform-load-data-into-unity-catalog.summary
+title: Summary
+metadata:
+  title: Summary
+  description: "Summary"
+  ms.date: 12/07/2025
+  author: weslbo
+  ms.author: wedebols
+  ms.topic: unit
+  ai-usage: ai-generated
+durationInMinutes: 2
+content: |
+  [!include[](includes/10-summary.md)]

learn-pr/wwl-databricks/cleanse-transform-load-data-into-unity-catalog/10-summary.yml

@@ -0,0 +1,14 @@
+### YamlMime:ModuleUnit
+uid: learn.wwl.cleanse-transform-load-data-into-unity-catalog.profile-data-summary-statistics
+title: Profile data
+metadata:
+  title: Profile Data
+  description: Learn how to profile data to generate summary statistics and assess data distributions in Azure Databricks using SQL commands and the data profiling feature.
+  ms.date: 12/07/2025
+  author: weslbo
+  ms.author: wedebols
+  ms.topic: unit
+  ai-usage: ai-generated
+durationInMinutes: 6
+content: |
+  [!include[](includes/2-profile-data-summary-statistics.md)]