Variables in the adf pipeline provide the functionality to temporary hold the values. They are used for similar reason like we do use variables in the programming language. Variables are available inside the pipeline and it can be set from inside the pipeline.
Set Variable and append variable are two type of activities used for the setting or manipulating the variables values.
There are two types of the variable :
System variable
User Variables
Copy activity Azure data factory with example
Azure Fundamentals Exam AZ- 900
System variable: These are some kind of the fixed variable from the azure pipeline itself. For example pipeline name, pipeline id, trigger name etc. You mostly need this to get the system information which might be needed in your use case.
You can also use them directly by clicking on add dynamic content A text window will appear and then you can use from drop down directly.
Once you click on variable name from below they will automatically inserted in the dynamic content box. Like this :
User variable : User variable is something which you declared manually based on your logic of the pipeline. For creating the variable you go the pipeline and under the variable tab click on + sign to add variable.
variable can be of three type. You decide based on your need. You can also provide the default value as per you need.
Variable types :
Array, boolean String.
Once the variable has been declared you can use them using :
@variable(‘variable_name’)
Contents
How to set variable in Azure Data Factory (ADF) using Set Variable Activity
Set variable Activity :
Provide the activity name and click on variable tab it will look like this :
It will ask you to select the variable from the drop down. Once you select the variable from drop down, a another text box appear to set the value :
Now in the value section you can add the fixed value or the dynamic content based on your need.
For example in below I am trying to set the variable as trigger runtime.
This is how I can access variable value in the activity :
Recommendations
Most of the Azure Data engineer finds it little difficult to understand the real world scenarios from the Azure Data engineer’s perspective and faces challenges in designing the complete Enterprise solution for it. Hence I would recommend you to go through these links to have some better understanding of the Azure Data factory.
Azure Data Engineer Real World scenarios
Azure Databricks Spark Tutorial for beginner to advance level
Latest Azure DevOps Interview Questions and Answers
You can also checkout and pinned this great Youtube channel for learning Azure Free by industry experts
Final Thoughts
Variables plays vital role in the Azure Data factory pipeline. Though it is not necessary that every pipeline should contains some variable. It is the business requirement and logic that will decide whether a variable needs to be included in the adf pipeline or not.
Deepak Goyal is certified Azure Cloud Solution Architect. He is having around decade and half experience in designing, developing and managing enterprise cloud solutions. He is also Big data certified professional and passionate cloud advocate.