Copy activity Azure data factory with example

Copy activity is one of the most popular and highly used activity in the azure data factory.

Copy activity is basically used for ETL purpose or lift and shift where you want to move the data from one data source to the other data source. While you copy the data you can also do the transformation for example you read the data from csv file which contains 10 columns however while writing to your target data source you want to keep only 5 columns. You can transform it and you can send only the required number of columns to the the destination data source.

For creating the copy activity you need to have your source and destination ready. Here destination is called as sink. Copy activity requires:

  1. Linked service
  2. Datasets

Assume you already have a linked service and data service created in case not you can please refer these links to create link service and datasets

How to create Linked service in Azure data factory

What is dataset in azure data factory

Step 1 : Go to the author tab and click on Plus sign to create the pipeline.

azure data factory copy activity

Step 2 : Give pipeline some name

azure data factory copy activity

Step 3: Go to activity blade and type the ‘copy’ in the activity search box. You will get the activity , drag the copy activity to the pipeline

azure data factory copy activity

Now you can see ‘red 1’ on the source and sink tab in the bottom. This is indicating that these two field need to be completely filled,

azure data factory copy activity

Let click on the source tab now it is asking to select some data set from the drop down if you have a data set already there you can select from the drop down or else you can click on the Plus sign to create a new data set.

What is dataset in azure data factory (Follow this to create the data set for blob)

Create dynamic dataset in ADF for Azure Sql Db ( Follow this to create the dataset for the azure sql db)

We have a csv file which contains the data which we want to load it into the azure sql database. Csv file look like this :

You can download the csv file from here :

We are using the sample table comes while creating the azure sql db. If you don’t have one you can create like this :

azure data factory copy activity

Now coming back again to our pipeline. In source tab lets select the azure blob data set pointing to our csv file location :

Ensure that for above data set First row as header should be set at true. Because our csv file has first line as header.

Lets select the Sink now. Here sink is our dataset which point to the sql table. I already have dataset ready. You can use this link to create : Create dynamic dataset in ADF for Azure Sql Db

In the dataset select the table name

Your pipeline is almost ready now. Just click on the debug and see if its working successfully.

You can 450 rows has been inserted into the table.

Leave a Comment