In this article, we will learn how to integrate the Azure Data Factory pipeline with git hosted in Azure DevOps.
- We need to create an Azure Data Factory service
- We need to create the following Azure DevOps components
- Create or use an existing Organization / Account
- Create or use an existing Project
- Create or use an existing Repository
Below are the three different ways in which you can integrate Azure Data Factory with git.
After choosing any one of the above methods, you will be taken to the below popup where you need to configure all the settings related to git integration.
|Repository Type||As of this writing, ADF supports git hosted in the following
|Azure DevOps Account||For every Organization that you work with, you will have one Azure DevOps Account / Organization created. Choose an existing one or create a new one.|
|Project Name||Choose an existing Project or a new one.|
|Git repository name||Choose an existing Repository or a new one.|
|Collaboration Branch||Default – master. Feel free to change it to any other branch.|
|Root Folder||Path of the folder within the repository where the JSON of the Data Factory components are saved.|
|Import existing Data Factory resources to repository
|If you have already authored the pipeline then you can check this checkbox which will import the JSON of the components into the branch specified in the Branch to Import resources into field|
|Branch to Import resources into||This field is visible only if Import existing Data Factory resources to repository is checked. Otherwise, this field is not visible.|
Once you specify all the fields mentioned in the above table in the popup, it should look something as shown below.
Review all the details in the above screenshot and click on Apply button. Once you click on the Apply button, you will be taken to the below screenshot where you can view all the setting applied and you can also Edit the information or even get disconnected with the Source Control settings.
And, if you navigate to the Azure DevOps repository (in my case it is adf-git-integration-project) then you would see an empty branch master is created as shown below.
That’s it. You have learnt 3 ways to Integrate ADF Pipeline with Azure DevOps – Git
Points to Ponder:
- We can link multiple ADF pipelines with the same Repository.
- We can have only one Collaboration branch (master branch is commonly used as collaboration branch)
- We can’t have a separate collaboration branch per environment. One common practice in applications is to have separate branches per environment.
In real-time projects, developers need to work collaboratively as team. You can learn how to achieve that while developing ADF pipelines by reading my article at Azure Data Factory – Collaborative development of ADF pipelines using Azure DevOps – Git
Do you like this article? If you want to get more updates about these kind of articles, you can join my Learning Groups