Are you looking to find how you can use the failure activity within the Azure data factory or maybe you are looking for a solution to the scenario where certain condition not reached then you manually wanted to set the pipeline as the Failed one with some user defined error code and error message. Then your wait has been over as Microsoft Azure Data factory has launched the Fail Activity using which you can purposefully make the pipeline failed. In this article, I will take you through a step-by-step process of creating the pipeline and using the Fail activity which will check the business condition and if it not meant it will make the pipeline failed with custom message. Let’s get into the details of the Azure Data Factory filter condition example.
Fail Activity is the activity in Azure Data Factory use to purposefully fail the pipeline along with sending the custom error code and error message with it.
For example you will use the Fail activity when you wanted to stop the pipeline and set the status as failed when certain condition not reached.
| Fail Activity | Description |
| Input | Error code and error message which can be hard coded or dynamic |
| Output | Error code and error message |
Contents
How to use Fail activity in the Azure data factory (ADF V2) pipeline with example?
- Go to the Azure data factory account and create one demo pipeline I am giving the name as fail-activity-demo pipeline. You can give any name as per your need or you may be using your existing pipelines.
- Go to the activity search box and type fail and in the result section, you will see the Fail activity. Just drag and drop this fail activity into the adf pipeline designer tab.
- Under the setting tab of Fail activity you can see two properties one is error code and another one is error message. In the error code property, you have to give your customized error code i.e. you can provide error code based on your need like 200, 404, 500 or any other like E100. For example I have given 500 as the error code. You can also give dynamic error code instead of hard coded one using the dynamic web content
- In the error message you can give customized error message as per your business logic and need. This can also be dynamic using the dynamic web content. For example I have given the error message as : ‘Internal Server Error Occurred by Azurelib.com Contact Administrator’
Our pipeline is ready to run just go to the debug tab. Click on debug this will execute the pipeline. If you have followed all the steps mentioned above your pipeline will fail successfully and get completed.
Once you go to the output tab and see the output of the fail activity you will must be able to see the error code and the error message which you have set in the Fail activity.
How to use Fail activity in the Azure data factory (ADF V2) pipeline with If Activity example?
- Go to the Azure data factory account and create one demo pipeline I am giving the name as fail-activity-demo-2 pipeline. You can give any name as per your need or you may be using your existing pipelines.
- Go to the activity search box and type if and in the result section, you will see the If Activity. Just drag and drop this if activity into the adf pipeline designer tab.
- Open the activities tab of the IF activity. Under the expression text box type in the condition. For example I am just putting the true as the condition. You can put in some more complex condition based on the business need.
- No click on the True activities tab and inside it we will add the fail activity.
- Go to the activity search box and type fail and in the result section, you will see the Fail activity. Just drag and drop this fail activity into the adf pipeline designer tab.
- Under the setting tab of Fail activity you can see two properties one is error code and another one is error message. In the error code property, you have to give your customized error code i.e. you can provide error code based on your need like 200, 404, 500 or any other like E100. For example I have given 200 as the error code. You can also give dynamic error code instead of hard coded one using the dynamic web content
- In the error message you can give customized error message as per your business logic and need. This can also be dynamic using the dynamic web content. For example I have given the error message as : ‘Internal Server Error Occurred by Azurelib.com Contact Administrator’
Our pipeline is ready to run just go to the debug tab. Click on debug this will execute the pipeline. If you have followed all the steps mentioned above your pipeline will fail successfully and get completed.
Once you go to the output tab and see the output of the fail activity you will must be able to see the error code and the error message which you have set in the Fail activity.
Use case Scenarios of using Fail Activity
- You may expecting some file to be available in the blob storage location to process. However if the file wasn’t there due to any reason then your pipeline will fail abruptly. By using the Fail activity here you can fail the pipeline in respectful manner by giving meaningful error code and error message.
- Assume that you have added the email notification for fail pipeline. In this case you can use the Fail activity smartly to receive the custom error message as part of email notification.
Microsoft Azure Data Factory Official Documentation Link
Recommendations
Most of the Azure Data engineer finds it little difficult to understand the real world scenarios from the Azure Data engineer’s perspective and faces challenges in designing the complete Enterprise solution for it. Hence I would recommend you to go through these links to have some better understanding of the Azure Data factory.
Azure Data Engineer Real World scenarios
Azure Databricks Spark Tutorial for beginner to advance level
Latest Azure DevOps Interview Questions and Answers
You can also checkout and pinned this great Youtube channel for learning Azure Free by industry experts
Final Thoughts
By this, we have reached to the last section of our article. So in this article we have learned what is the fail activity in adf, how you can use the fail activity in the Azure data factory pipeline. we also talked about what are the various real world use cases and scenarios where you will use filter activity along with getmetadata activity. I hope now you are conceptually clear on how you can use the fail activity and solve some of your business use cases.
Please share your comments and suggestions in the comment section below and I will try to answer all your queries as time permits.
Deepak Goyal is certified Azure Cloud Solution Architect. He is having around decade and half experience in designing, developing and managing enterprise cloud solutions. He is also Big data certified professional and passionate cloud advocate.