Azure Data Factory – 3 ways to Integrate ADF Pipeline with Azure DevOps – Git

In this article, we will learn how to integrate the Azure Data Factory pipeline with git hosted in Azure DevOps.

Prerequisites:

1. We need to create an Azure Data Factory service
2. We need to create the following Azure DevOps components
   • Create or use an existing Organization / Account
   • Create or use an existing Project
   • Create or use an existing Repository

Below are the three different ways in which you can integrate Azure Data Factory with git.

Method1:

Method2:

Method3:

After choosing any one of the above methods, you will be taken to the below popup where you need to configure all the settings related to git integration.

Component name	Description
Repository Type	As of this writing, ADF supports git hosted in the following   Github Azure DevOps (I will select this for this article)
Azure DevOps Account	For every Organization that you work with, you will have one Azure DevOps Account / Organization created. Choose an existing one or create a new one.
Project Name	Choose an existing Project or a new one.
Git repository name	Choose an existing Repository or a new one.
Collaboration Branch	Default – master. Feel free to change it to any other branch.
Root Folder	Path of the folder within the repository where the JSON of the Data Factory components are saved.
Import existing Data Factory resources to repository	If you have already authored the pipeline then you can check this checkbox which will import the JSON of the components into the branch specified in the Branch to Import resources into field
Branch to Import resources into	This field is visible only if Import existing Data Factory resources to repository is checked. Otherwise, this field is not visible.

Once you specify all the fields mentioned in the above table in the popup, it should look something as shown below.

Review all the details in the above screenshot and click on Apply button. Once you click on the Apply button, you will be taken to the below screenshot where you can view all the setting applied and you can also Edit the information or even get disconnected with the Source Control settings.

And, if you navigate to the Azure DevOps repository (in my case it is adf-git-integration-project) then you would see an empty branch master is created as shown below.

That’s it. You have learnt 3 ways to Integrate ADF Pipeline with Azure DevOps – Git

Points to Ponder:

• We can link multiple ADF pipelines with the same Repository.
• We can have only one Collaboration branch (master branch is commonly used as collaboration branch)
• We can’t have a separate collaboration branch per environment. One common practice in applications is to have separate branches per environment.

Next Steps:

In real-time projects, developers need to work collaboratively as team. You can learn how to achieve that while developing ADF pipelines by reading my article at Azure Data Factory – Collaborative development of ADF pipelines using Azure DevOps – Git

Do you like this article? If you want to get more updates about these kind of articles, you can join my Learning Groups

WhatsApp

Telegram