zbalachandar / slowly-changing-dimension-scd-type-1-using-adf-project-11

In this project, I discussed about Slowly changing Dimension Type1 using mapping data flows in azure data factory(ADF).

Slowly-Changing-Dimension-SCD-Type-1-Using-ADF-Project-11

In this project, I discussed about Slowly changing Dimension Type1 using mapping data flows in Azure Data Factory (ADF).

End to End Data-Pipeline-Project

Data Pipeline using Azure DataFactory

💾 Dataset

Dataset :

SQL script:

CREATE TABLE employee ( employee_id INT, FirstName NVARCHAR(128), salary INT, location NVARCHAR(128) )

INSERT INTO EMPLOYEE VALUES(1,'manish',10000,'india') INSERT INTO EMPLOYEE VALUES(2,'NEHA',5000,'india') INSERT INTO EMPLOYEE VALUES(4,'SURYA',5000,'UK')

Business Requirement.

Design and Development of pipeline:

Project steps to follow:

what we have covered in this project:

• We have Covered End to End ADF Pipeline Project
• Developed the Data Pipeline according to the business requirement

⚙️ Data Pipeline

Create the resources for Data pipeline and processing using AZURE DATA FACTORY.

Azure SQL Server:

Create the Azure Data Factory:

Create Source point:

Dataflow :

Debug:

Data Preview:

pipeline 1:

pipeline 2,3:

Publish all the Activities and dataflow:

Run pipelines:

Desired Output - SQL SCD operation :

🛠️ Technologies Used

• Data Pipeline: Azure Data Factory
• Data Loading: Azure SQL Server, Database.

📋 Credits

• This Project is inspired by the video of the YouTube Channel "Learn by doing it"

📨 Contact Me

LinkedIn • Gmail •