Triggers in Azure Data Factory

In the last post we have seen Debug Azure Data Factory Pipeline. Debugging will execute the pipeline manually. However in the production environment you must don’t manual triggering of the pipeline. You would like to schedule automatic execution’s of pipeline. This scheduling of the pipeline is provided by the Triggers in the ADF. In this blog we will see what is triggers and how you can create triggers in adf.

In the top pane of the pipeline you will find the option of the

When you click on the Trigger, you can see the two options :

1. Trigger now
2. New / Edit

Trigger now : This will actually execute your pipeline right away and this will be for just that time only. It won’t schedule anything. You will generally use this ,when you just want to execute the pipeline once may be for initial one time load or may because it was failed earlier etc.

Difference between Trigger Now and Debug : There is subtle difference between the trigger now and the debug. Even though both will execute the pipeline once and that too immediately. Difference lies in the way the log get captured. When you debug, you can see the pipeline details in below output pane while when you trigger now, you can see logs in the monitor tab.

New /Edit Trigger : This opens the door for the pipeline run automation and scheduling. You can actually create the trigger of your own. In trigger you can provide when and at what frequency you want this trigger to get executed.

For example if you want to set at what time everyday your pipeline runs or you want to set at which date of month or may be on which day of the week, your pipeline runs. You can set based on all these parameters. Triggers provides immense option to schedule the pipeline runs.

The time zone it uses to schedule is the UTC time zone. You can also define when actually you want to stop this schedule run. You can set the end date for the trigger as well. After which this trigger will not execute ant pipeline.

Trigger also have start date as well. Sometimes you have create the pipeline way ahead and early and you want to start executed it after may be next 15 days. You can do that by setting the start day for the trigger.

Lets create the triggers step by step and see how you can do it on your own later on.

Click on Add Trigger and after that click on the New/Edit :

You will see a dropdown to choose trigger from. You can choose any existing one to edit or click on +New to create new one.

Lets click on +New to create new trigger. You will see the next blade like this:

Provide the trigger name and description ( this is optional, generally it is good practice to give description which mentioned its purpose)

There could be three type of the trigger : Schedule/ Tumbling window / Event

Schedule : In the schedule type you want to set trigger run in based on some specific time frequency like :

You can set :

1. Start date
2. Time Zone
3. Recurrence ( How you want to repeat the trigger)
4. In advance section you can select the execution time. (Like you may be wanted to schedule it mid night every day)

Besides this you set the end date for this trigger. After that it wont become active:

To activate trigger just select the option Yes and click the ok.

Trigger wont starts its work until and unless it is set as activated true.

Once the trigger added in the pipeline you can see number of trigger count in the top pane like this :

Event Trigger : You can also have event based create triggers in adf. Azure data factory provides the event based trigger through the blob storage. You can create the trigger in such a fashion that it will get trigger once there is any blob created or deleted in specified container.

A trigger can be map to many pipeline. Assume that you have set the schedule using the trigger and you want to run couple of more pipeline in same schedule manner. You can use the same trigger and add it to other pipelines.

How many types of triggers are there in Azure Data Factory?

There are four types of triggers available in the Azure Data Factory

1. Schedule

2. Tumbling Window

3. Storage Events

4. Customs Events

Summary : This is how you can create triggers in adf and set up for your pipelines. We have also learned the various types of triggers and their scheduling. Difference between the debug and the trigger as well. Hope this has been enlightening read on triggers. In the next blog I will try to talk to about monitoring the azure pipelines.