Variables in ADF Pipeline

Variables in the adf pipeline provide the functionality to temporary hold the values. They are used for similar reason like we do use variables in the programming language. Variables are available inside the pipeline and it can be set from inside the pipeline.

Set Variable and append variable are two type of activities used for the setting or manipulating the variables values.

There are two types of the variable :

System variable

User Variables

Copy activity Azure data factory with example

Azure Fundamentals Exam AZ- 900

System variable: These are some kind of the fixed variable from the azure pipeline itself. For example pipeline name, pipeline id, trigger name etc. You mostly need this to get the system information which might be needed in your use case.

Variables in Adf pipeline

You can also use them directly by clicking on add dynamic content A text window will appear and then you can use from drop down directly.

Variables in Adf pipeline
Variables in Adf pipeline

Once you click on variable name from below they will automatically inserted in the dynamic content box. Like this :

Variables in Adf pipeline

User variable : User variable is something which you declared manually based on your logic of the pipeline. For creating the variable you go the pipeline and under the variable tab click on + sign to add variable.

Variables in Adf pipeline

variable can be of three type. You decide based on your need. You can also provide the default value as per you need.

Variable types :

Array, boolean String.

Once the variable has been declared you can use them using :

@variable(‘variable_name’)

How to set variable in Azure Data Factory (ADF) using Set Variable Activity

Set variable Activity :

Provide the activity name and click on variable tab it will look like this :

It will ask you to select the variable from the drop down. Once you select the variable from drop down, a another text box appear to set the value :

Now in the value section you can add the fixed value or the dynamic content based on your need.

For example in below I am trying to set the variable as trigger runtime.

This is how I can access variable value in the activity :

Recommendations

Most of the Azure Data engineer finds it little difficult to understand the real world scenarios from the Azure Data engineer’s perspective and faces challenges in designing the complete Enterprise solution for it. Hence I would recommend you to go through these links to have some better understanding of the Azure Data factory.

Azure Data Factory Insights

Azure Data Engineer Real World scenarios

Azure Databricks Spark Tutorial for beginner to advance level

Latest Azure DevOps Interview Questions and Answers

You can also checkout and pinned this great Youtube channel for learning Azure Free by industry experts

IT Skills Upgrade – YouTube

Final Thoughts

Variables plays vital role in the Azure Data factory pipeline. Though it is not necessary that every pipeline should contains some variable. It is the business requirement and logic that will decide whether a variable needs to be included in the adf pipeline or not.

Leave a Comment