Microsoft Azure Storage and Database Part 13 – Azure Blob Storage – Work With AZCopy Tool

Hello Everybody,

Hope you all are doing good !!! 🙂 .

In one of our previous article we have discussed, how to Host a static website on Azure. Today in this article we will discuss how to configure and use AZCopy tool.

Previous Azure series :

1. Learn Basics Of Azure Networking In 60 Hours
2. Learn Basic Of Azure Active Directory And Azure Identity And Access Management
3. Azure DevOps – Learn at one place

If you have missed our previous articles on Azure Storage and Database Series, please check it in following links.

Part 1 – Overview Of Azure Storage and Database

Part 2 – Azure Storage Account

Part 3 – Azure Blob Storage

Part 4 – Work With Azure Blob Storage

Part 5 – Storage Explorer For Azure Storage

Part 6 – Azure Blob Storage – Snapshot Using Storage Explorer

Part 7 – Azure Blob Storage – Shared Access Signature (SAS)

Part 8 – Secure Azure Storage Using Stored Access Policy

Part 9 – Secure Azure Storage Using RBAC

Part 10 – Configure Stored Access Policy Using PowerShell

Part 11 – Get Shared Access Signature (SAS) Using PowerShell

Part 12 – Azure Blob Storage – Host Static Website In Azure Storage Account

Next Article : Part 14 – Azure File Service – Overview

AzCopy Tool :

AzCopy is a command-line tool that is used to upload and download blobs/files from or to the Azure Blob Storage. The AzCopy tool is a free and easy to use tool that allows us to copy and move data to and from Azure storage. This command-line utility is not required to be installed on the workstation. It’s a great command-line utility that can automate and streamline the process but requires some setup. We can download the appropriate executables from the below location. The executable files are compressed in .zip (Windows) or .tar (Linux). We can download and unzip the executables before use it.

• Microsoft Windows 64 Bit
• Microsoft Windows 32bit
• Linux
• macOS

Prerequisites :

In this article, we are going to learn how to prepare our windows system to use AzCopy. This includes downloading and authenticating the tool to have access to Azure storage but before we proceed with this, we need to make sure we have the following prerequisites.

• Windows PowerShell 5.1 or later version
• We need one Azure subscription. If we do not have one, we can request for a trial subscription.
• We need one Azure Storage Account. If we do not have one, we can create by following this article.
• We need An Azure AD tenant created with a user account part of the Storage Blob Data Contributor or Storage Blob Data Owner groups

Let’s download the AzCopy tool, once we have all the above prerequisites. To download click on the above provided link Microsoft Windows 64 Bit and it will start downflowing the azcopy_windows_amd64_10.10.0.zip file s showing in the following figures.

Authenticating AzCopy :

Now we have downloaded our AZCopy.exe as shown in the above figure, before use AzCopy for any purpose ,we need to authenticate our Azure subscription to access our Azure Storage. followings are two way to authenticate AzCopy tool.

1. Azure Active Directory
2. Shared Access Signature (SAS) token

Azure Active Directory Authentication :

In this article we will see how to authenticate using the first approach Azure Active Directory authentication. We can provide AzCopy with authorization credentials by using Azure AD. So that we don’t need to append a shared access signature (SAS) token to each command. In this approach the second step is to choose one of the following types of security principal we want to authorize.

1. User Identity - User Identity is any user that has an identity in Azure AD. It’s the easiest security principal to authorize.
2. Managed identities - Managed Identity is better suited for scripts that run from an Azure Virtual Machine (VM).
3. Service Principal - Service Principal is better suited for scripts that run on-premises.

In our case we will use the first option User Identity and to read more about other approach check this Microsoft Article. Now let’s check if our user identity has one of the following role assignment which is required to work with blob storage like upload a new blob.

1. Storage Blob Data Contributor
2. Storage Blob Data Owner

Let’s go through the following steps, once we are ready with all of our above requirement .

Step 1 – open the command prompt and run following command to Authorize our user identity. Here I have provided the Tenant Id of the Azure AD where my user exists.

azcopy login –tenant-id “323jjkh34-rrwekbqjknkl**********”

Step 2 – As showing in the above figure, this command returns an authentication code and the URL of a website. Open the website, provide the code, and then choose the Next button and continue as per the guidance from the browser as showing in the following figures.

Now we can use our AzCopy tool for our requirement. In this section, we will do some lab exercise with few tasks.

Upload a File : Upload a file by using the azcopy copy command as showing in the following example and figure.

Syntax : azcopy copy '<local-file-path>' 'https://<storage-account-name>.<blob or dfs>.core.windows.net/<container-name>/<blob-name>'

azcopy copy “C:\MANAS DATA\AZURE\Azure_AZ-104\Azure Storage And Database Services\AZCopy\7 azcopy code to login.png” “https://kj21storageacount.blob.core.windows.net/kj-container/AzCopy Authorization.png“

Upload a directory : It copies a directory (and all of the files in that directory) to a blob container. The result is a directory in the container by the same name by using the following command as showing in the following figures.

Syntax : azcopy copy '<local-directory-path>' 'https://<storage-account-name>.<blob or dfs>.core.windows.net/<container-name>' --recursive

azcopy copy “C:\MANAS DATA\AZURE\Azure_AZ-104\Azure Storage And Database Services\AZCopy” “https://kj21storageacount.blob.core.windows.net/kj-container” –recursive

Download a directory : Download a directory by using the following command as showing in the following figures.

Syntax : azcopy copy 'https://<storage-account-name>.<blob or dfs>.core.windows.net/<container-name>/<directory-path>' '<local-directory-path>' --recursive

azcopy copy “https://kj21storageacount.blob.core.windows.net/kj-container/Knowledge Jinction” “C:\Users\moharman\Desktop” –recursive

We can do much more than what we saw in the above section, if you want to know more on AzCopy, read this Microsoft documentation.

Copy Files Options :

Using AzCopy, we can copy the files from the source to the destination. It supports the following directions:

1. local <-> Azure Blob (SAS or OAuth authentication)
2. local <-> Azure Files (Share/directory SAS authentication)
3. local <-> Azure Data Lake Storage Gen 2 (SAS, OAuth, or shared key authentication)
4. Azure Blob (SAS or public) -> Azure Blob (SAS or OAuth authentication)
5. Azure Blob (SAS or public) -> Azure Files (SAS)
6. Azure Files (SAS) -> Azure Files (SAS)
7. Azure Files (SAS) -> Azure Blob (SAS or OAuth authentication)
8. Amazon Web Services (AWS) S3 (Access Key) -> Azure Block Blob (SAS or OAuth authentication)
9. Google Cloud Storage (Service Account Key) -> Azure Block Blob (SAS or OAuth authentication) [Preview]

I hope this is informative to you. Please let me know if I missed anything important or if my understanding is not up to the mark.

Next Article : Part 14 – Azure File Service – Overview

Keep reading, share your thoughts, experiences. Feel free to contact us to discuss more.

If you have any suggestion / feedback / doubt, you are most welcome. Stay tuned on Knowledge-Junction, will come up with more such articles.

Thanks for reading 🙂 .