DP-700: Microsoft Fabric Data Engineer

Course Description:

Overview

• Explore compute and storage options for analytical workloads
• Design and implement serving layers with star schemas and CDC
• Ingest data using Spark, Data Factory, Azure Synapse pipelines
• Transform and explore data interactively in Spark and Databricks
• Monitor and optimize data loads and query performance
• Secure data at rest and in transit within Azure Synapse
• Create dashboards and predictive models using Synapse Studio
• Support real time streaming and HTAP with Synapse Link

Objectives

• Understand Azure Fabric compute and storage technologies
• Implement batch and streaming data pipelines
• Design analytical serving layers and data lake refinement
• Use Spark and serverless SQL for data exploration
• Monitor system health and optimize performance
• Apply security best practices in Synapse Analytics
• Build dashboards and integrate predictive analytics
• Implement HTAP scenarios with Synapse Link

Target Audience

• Data Engineers and Architects
• Business Intelligence Professionals
• Data Analysts and Data Scientists

Module 1: Explore compute and storage options for data engineering workloads

• Introduction to Azure Synapse Analytics and Databricks
• Overview of Azure Data Lake Storage and Delta Lake
• Stream Analytics for real–time data
• Lab: Explore compute and storage options
• Combine streaming and batch in a single pipeline
• Organize data lake into refinement levels
• Index data lake files for query acceleration

Module 2: Design and implement the serving layer

• Design a star schema for analytical workloads
• Use Azure Data Factory for code-free transformations
• Populate slowly changing dimensions
• Lab: Designing and implementing the serving layer

Module 3: Data engineering considerations for source files

• Design modern data warehouse with Synapse Analytics
• Secure data warehouse and data lake files
• Lab: Data engineering considerations

Module 4: Run interactive queries using Synapse serverless SQL pools

• Query Parquet and CSV files with T-SQL
• Create metadata objects and external tables
• Secure access via RBAC and ACLs
• Lab: Run interactive queries using serverless SQL pools

Module 5: Explore, transform, and load data using Apache Spark

• Big data engineering with Spark in Synapse Analytics
• Ingest and transform data with Spark notebooks
• Integrate SQL and Spark pools
• Lab: Explore, transform, and load data using Spark

Module 6: Data exploration and transformation in Azure Databricks

• Use DataFrames for data exploration and filtering
• Perform advanced DataFrame operations
• Lab: Data exploration and transformation in Databricks

Module 7: Ingest and load data into the data warehouse

• Petabyte-scale ingestion with PolyBase and COPY
• Workload management in Synapse pipelines
• Lab: Ingest and load data into the data warehouse

Module 8: Transform data with Azure Data Factory or Synapse pipelines

• Build data integration pipelines and mapping data flows
• Lab: Transform data with Data Factory or Synapse pipelines

Module 9: Orchestrate data movement and transformation

• Create linked services and orchestrate via Synapse Pipelines
• Lab: Orchestrate data movement with Synapse Pipelines

Module 10: Optimize query performance with dedicated SQL pools

• Use developer features and data loading best practices
• Optimize data warehouse query performance
• Lab: Optimize query performance with dedicated SQL pools

Module 11: Analyze and optimize data warehouse storage

• Analyze table space and columnstore details
• Use materialized views and minimally logged operations
• Lab: Analyze and optimize data warehouse storage

Module 12: Support HTAP with Azure Synapse Link

• Configure Synapse Link for Cosmos DB
• Query Cosmos DB analytical store with Spark and SQL
• Lab: Support HTAP with Azure Synapse Link

Module 13: End-to-end security with Azure Synapse Analytics

• Secure workspace, firewall, and Key Vault integration
• Implement column-level and row-level security
• Lab: End-to-end security with Azure Synapse Analytics

Module 14: Real-time stream processing with Stream Analytics

• Process streaming data with Event Hubs and Stream Analytics
• Use windowing functions and scale Stream Analytics jobs
• Lab: Real-time stream processing with Stream Analytics

Module 15: Create a stream processing solution with Event Hubs and Databricks

• Structured streaming in Databricks with sliding windows
• Implement watermarking and Event Hubs integration
• Lab: Create stream processing solution with Event Hubs and Databricks

Module 16: Build reports with Power BI integration

• Create reports and optimize them with materialized views
• Visualize data via serverless SQL and Power BI
• Lab: Build reports using Power BI integration

Module 17: Perform integrated machine learning in Synapse Analytics

• Create AML linked services and trigger AutoML experiments
• Enrich data with trained models and serve predictions in Power BI
• Lab: Perform integrated machine learning processes

We offer live-instructor group training classes for your team.

Upskill your team from the comfort of your office or online.

Benefits of Group Training Include:

• Experienced Professional Instructor Trains Your Team
• Content Focused On Your Team's Needs
• Convenient Scheduling and Class Setup
• Significant Per/Student Cost Savings
• Online, On-Site and Blended Options Available

Get A Free Consultation