Azure Data Factory – All about publish branch adf_publish

In the Azure Data Factory – Collaborative development of ADF pipelines using Azure DevOps – Git  article, we have learned how to collaborate with different team members while working with Azure Data Factory.
Each developer creates an individual branch for each of their tasks as shown below.

In the above screenshot, you have Task1 and Task2 branches that were created for two different tasks. And, you also see the other two branches.

1. master – master is the collaboration branch that is used to merge the code developed by all the developers.
2. adf_publish – this branch is specific to Azure Data Factory which gets created automatically by the Azure Data Factory service. The adf-publish branch, as the name suggest, it contains the code, specifically, the json code related to all the ADF pipeline and it’s components that are published to the Data Factory service.

Please make sure you publish all the work that you have done in developing the ADF pipeline. If you have not yet published all your work then you might see something as shown below.

The above screenshot is a version which has the code related to Linked Services. Linked Services will get published to the adf_publish branch immediately (only when you integrate ADF Service with git) after they are created.
And, when you want to publish all the work, then you need to click on the Publish button which will show a popup for confirmation as shown below.

Once, you click on the Ok button, It will generate the ARM templates, saves them into the adf_publish branch and then all the ADF components will be published to the ADF Instance. Below are the notifications that you will see.

And, If you have published all the work, then the adf_publish branch should contain all the files as shown below.

ARMTemplateForFactory.json – The ARM template file that consists of ALL the resources that we have in the ADF pipeline.

ARMTemplateParameterForFactory.json – The parameters that the ARM Template would need. The parameter files contains all the names and configurations of the services that are environment specific.

FAQs about adf_publish branch in Azure Data Factory.

1. What is adf_publish?
   1. It is a special branch that get’s created automatically.
2. What does it contain?
   1. It contains the code (in json format) of all the services that are published to Azure Data Factory service.
3. Why is adf_publish branch is empty?
   1. The adf_publish branch would contain files in the below scenarios
      1. When you create Linked Services, they will be published immediately
      2. When you publish the Pipeline, all it’s components will be created in the adf_publish branch.
   2. I have not published anything yet why do I see some JSON code already created?
      1. You might have created Linked Services which will be published immediately after you create them.
   3. What is the folder named PartialArmTemplates inside the adf_publish branch?
      1. The PartialArmTemplates folder contains multiple json files each created for individual Linked Services that you create.
   4. Can I use some other branch to save published contents instead of adf_publish?
      1. By default, adf_publish is the default Publish Branch. However you can use any of your branches as the Publish branch.

Do you like this article? If you want to get more updates about these kind of articles, you can join my Learning Groups

WhatsApp

Telegram