Azure Data Engineering for developers
Training highlights:
- Prepared training contents for ETL/ Database developers
- Design principles
- Focus on what and why over how
- Complete hands-on (implementation of mini project)
- ETL/Data developers
- Data architects
- Data/ETL Trainers / professors
- Technical Project Managers
Introduction to Azure
1. Introduction to Azure Cloud
2. What is difference between Azure Cloud and On-Premises
3. What is Subscriptions and Resource Groups
4. Different offerings of Cloud IaaS, PaaS and SaaS
5. Variety of Azure services related to Data management
Sample Architecture of Big Data on Azure
1. Batch load and transform data from Cloud to Azure
2. Batch load and transform data from On-Prem to Azure
Introduction to Storage Services
3. Structure, Scalability, Performance, Usage
4. What is Azure Storage and Azure Data Lake Store Gen 2
5. [Hands-on] How to create and manage
6. Difference between Blob and ADLS
7. [Hands-on] Using Azure Storage Explorer
8. [Hands-on] Managing security using managed policies and SAS urls
Introduction to Azure SQL
1. Introduction to Azure SQL Database
· Performance and scalability
· Flavors
· Security features
· Elastic pools
2. Introduction to Azure SQL Data Warehouse
· Performance and scalability
· Security features
· Supported distribution and sharding patterns
3. Hands on:
· Creating SQL DB & DW
· Scaling
· Implementing Dynamic Data Masking
4. Introduction of Polybase :
· [Hands-on] Implementing polybase using External sources and tables
· [Hands-on] Loading different kinds of data from various formats (CSV,
Parquet, JSON and other Hadoop formats) from external storages
Azure Data Factory
1. Concepts – ADF - an orchestration tool, components, supported connectors
2. Pipelines, Datasets, Triggers and Integration runtime
3. [Hands-on] : Creating ADF
4. [Hands-on] : Creating Datasets and Pipelines – load data Azure to Azure
5. [Hands-on] : Installation of Self Hosted Integration Runtime – load data onprem
to cloud
6. [Hands-on] : load data from web to Azure
7. [Hands-on] : Activities: ForEach, Lookup, IfCondition, Copy, Stored Procedure
8. [Hands-on] : Activities: Monitoring in detail
Azure Databricks
1. Introduction to Spark
b. Architecture
c. Spark APIs – Dataframe, Dataset, SparkSQL
d. Panda
2. What is Azure Databricks and why to use it (including Delta lake)
3. [Hands-on] : Create and manage cluster
4. [Hands-on] : Create and work on notebooks using Python
5. [Hands-on] : Managing notebooks
6. [Hands-on] : Read data from storage services and process (transform) it
7. [Hands-on] : SparkSQL (includes temp tables and views)
8. [Hands-on] : Databases, Tables, Delta tables
9. [Hands-on] : Connect to SQL DB and ingest data
10. [Hands-on] : Installation of libraries from Wheel, Jar, Maven, PyPi
Note:
i) Basic knowledge of Azure will speed up training and will focus on core contents. Please visit bigdatacloud.in to know basics of data services by Azure.
ii) Knowledge of database (Data Warehouse concepts) and data processing is desired
iii) This training will help you to prepare for DP 200 & DP 201
Total Max hours : 30 Hours.
3. [Hands-on] : Create and manage cluster
4. [Hands-on] : Create and work on notebooks using Python
5. [Hands-on] : Managing notebooks
6. [Hands-on] : Read data from storage services and process (transform) it
7. [Hands-on] : SparkSQL (includes temp tables and views)
8. [Hands-on] : Databases, Tables, Delta tables
9. [Hands-on] : Connect to SQL DB and ingest data
10. [Hands-on] : Installation of libraries from Wheel, Jar, Maven, PyPi
Note:
i) Basic knowledge of Azure will speed up training and will focus on core contents. Please visit bigdatacloud.in to know basics of data services by Azure.
ii) Knowledge of database (Data Warehouse concepts) and data processing is desired
iii) This training will help you to prepare for DP 200 & DP 201
Total Max hours : 30 Hours.