Tag: Azure Blob Storage

While delving into Azure file service I came across a requirement to allow clients to access files from file storage using a URL. At the time of writing you cannot use Shared Access Signatures with file storage to grant them access. I am going to show one possible solution using ASP.NET MVC that will allow you to use a URL like this:

http://sitename.azurewebsites.net/Storage/RootFolder/Inroot.png
http://sitename.azurewebsites.net/Storage/RootFolder/SubFolder/File.png

Where the section of the URL in bold will match the exact path in your Azure file share and map to a file, in this case:

https://accountname.file.core.windows.net/ShareName/RootFolder/Inroot.png

In the example above Storage is just a friendlier name I chose as my route to the controller.

Steps

1. In your MVC project create a new controller and add an action to it that will be used to retrieve the file:
   
public class StorageController : Controller
{
public ActionResult StreamFile(string filepath)
{
string storagebase =        "https://StorageAccount.file.core.windows.net/ShareName/";
var storageAccount = CloudStorageAccount.Parse(StorageConnectionString);
var fileclient = storageAccount.CreateCloudFileClient();
var fileUri = new Uri(storagebase + filepath, UriKind.Absolute);
var azfile = new CloudFile(fileUri, fileclient.Credentials);
return new FileStreamResult(azfile.OpenRead(), "image/png");
}
}


Note: I am streaming the file back, for scalability and performance reasons you do not want to read large files into memory.

2. Add a new route. In this case I am redirecting all the requests for Storage to my StorageController and the StreamFile action, everything after the Storage section of the URL will be passed along in the filepath parameter

routes.MapRoute(
name: "Files",
url: "Storage/{*filepath}",
defaults: new {controller = "Storage", action = "StreamFile", filepath = ""});


3. Enable runAllManagedModulesForAllRequests in your web.config

<system.webServer>
<modules runAllManagedModulesForAllRequests="true">
...
</system.webServer>


The result

We need runAllManagedModulesForAllRequests to enable ASP.NET to service the request instead of IIS. Keep in mind this is a demo and not meant for high performance sites. Enabling runAllManagedModulesForAllRequests is bad for performance since all managed modules will run for all requests.

Update: If you can’t or don’t want to change the code for your file access mechanism or you absolutely have to access the files using an SMB share you could move your application to an Azure Web Role which supports mounting a file share.

• About
• Latest Posts

Francois Delport

I am a cloud and devops consultant, technology fan and previously a professional C# developer with a keen interest in system design and architecture. Currently I am involved in projects using Azure, the Microsoft stack and DevOps. I am based in Melbourne, Australia. Email: [email protected]

Latest posts by Francois Delport (see all)

• How To Convert A Linux Disk Or Image File To VMDK File - July 8, 2020
• Read SMART Attributes Using PowerShell And Smartmontools - February 10, 2020
• How To Install OMS Agent On Server Core - June 25, 2018

In my previous post I discussed how to create your file share and add some files to it using Powershell. In this post I will show you how to use the Azure Storage API to manipulate your file share.

Scenarios where this would be useful.

• On-premise applications.
• Accessing the share from Azure but in different regions than the one where your share is residing.
•  Using the share in scenarios where you cannot mount file shares like Azure websites.

I am going to explore the first scenario and change one of my existing on-premise applications to use Azure file service instead of the local file system. This is actually a quick change since the application performs file operations using a file repository class not the System.IO classes directly. I initially did this to make the code unit testable and now it is also making it easy to swap out my current implementation for the Azure file service one.

The Code
The easiest way to install the Windows Azure Storage client library is using the Nuget package manager console:

Install-Package WindowsAzure.Storage


It can also be obtained when you install Azure SDK for .NET, which I can recommend if you are going to do any Azure development. After installing add the following imports:

using Microsoft.WindowsAzure.Storage;
using Microsoft.WindowsAzure.Storage.File;

Connecting to your storage account
var conString = "DefaultEndpointsProtocol=https;
AccountName=YourAccountName;AccountKey=YourStorageAccountAccessKey";
var storageAccount = storageAccount.Parse(conString);
var fileClient = storageAccount.CreateCloudFileClient();

Creating a new  share
var newShare = fileClient.GetShareReference("NewShareName");
newShare.CreateIfNotExistsAsync().Wait();


Listing the contents of existing shares
var ExistingShare = fileClient.GetShareReference("ExistingShareName");
shareRoot = ExistingShare.GetRootDirectoryReference();
var contents = shareroot.ListFilesAndDirectories();
var existingfile = shareRoot.GetFileReference("FileInRootDir");
var existingDirectory = shareRoot.GetDirectoryReference("DirectoryInRootDir");

Note contents is of type IEnumerable<IListFileItem> and contains files of type CloudFile as well as directories of type CloudFileDirectory both implement the IListFileItem interface.

Creating files and directories
shareRoot = fileshare.GetRootDirectoryReference();
shareRoot.GetFileReference("NewFileName").Create(NewFileSize);
shareRoot.GetDirectoryReference("NewDirectoryName").Create();

Downloading a file
//To a local file
existingfile.DownloadToFile("LocalFilePath", FileMode.Create);


Read a file
//Read the contents of a text file into a string
string fileContents = existingfile.DownloadText();
//Read bytes from a file into byte array
var fileContents = existingfile.DownloadToByteArray(byteArray,offset);

Retrieve a directory or file directly
As you can see it will be very cumbersome to walk down a shares’ directory hierarchy to retrieve files. You can gain access to a file or directory directly by using its URI property. It will look something like this:
https://YourAccount.file.core.windows.net/ShareRootDir/1stDir/SubDir/FileName
And you can use it in the following way:
var fileUri = new Uri(FullURI, UriKind.Absolute);
var azfile = new CloudFileDirectory(fileUri , fileClient.Credentials);

It might take a bit of time to wrap your head around the fact that the API is using the CloudFileDirectory type that represents both files and directories. If you want to work with files only you can list them using the following snippet:
SomeAzurefolder.ListFilesAndDirectories().OfType<CloudFile>()

The examples above are for demonstration only, in production you would be better of using async versions of these methods and streaming files up or down instead of reading whole files into memory.

• About
• Latest Posts

Francois Delport

I am a cloud and devops consultant, technology fan and previously a professional C# developer with a keen interest in system design and architecture. Currently I am involved in projects using Azure, the Microsoft stack and DevOps. I am based in Melbourne, Australia. Email: [email protected]

Latest posts by Francois Delport (see all)

• How To Convert A Linux Disk Or Image File To VMDK File - July 8, 2020
• Read SMART Attributes Using PowerShell And Smartmontools - February 10, 2020
• How To Install OMS Agent On Server Core - June 25, 2018

To get started with Azure file service it is necessary to join the preview first. It is very easy to sign up and it did not take very long to receive confirmation that the file service feature was enabled on my account.

Create storage account

The next step is to create a new storage account, you need a new one because storage accounts created before Azure file service was activated will not be able to create file shares. The easiest way create a new storage account is using the Azure portal .

You can use the current portal or the preview portal to create a storage account but the preview portal does not support file service yet and it won’t show under your storage endpoints.

Create A Share

At the time of writing it was not possible to create a share from the portal but it can be done using Powershell or the REST API. I will cover using the API later on so for now I will use Powershell. You can download Azure Powershell using the Microsoft Web Platform Installer.

PS: On my VM Azure Powershell did not work initially, it was missing some dlls and could not load the Azure Modules, in my case installing Azure SDK fixed the problem. Once installed you can launch it by typing “azure powershell” in the start menu.

Enter the following two commands to create a new share:
$ctx = New-AzureStorageContext storage_account_name storage_account_key
New-AzureStorageShare YourNewShareName -Context $ctx


In the code above storage_account_name is the name of the new storage account created earlier, just the name not the whole URL. Your storage_account_key can be retrieved from the portal by clicking Manage Access Keys under your storage account.

Once the share is created you have a few options to create directories and files in it.

• Mounting the share in a VM

You can only mount the share on VMs that are in the same region as the file share. In a Powershell window or command prompt execute the following:

net use z: \\storage_account_name.file.core.windows.net\YourShareName /u:storage_account_name storage_account_key

Alternatively you can store your account key on the VM that will be mounting the share, this way you can mount shares without specifying your key every time. In a Powershell window or command prompt execute the following two commands:

cmdkey /add:storage_account_name.file.core.windows.net /user:storage_account_name /pass:storage_account_key


net use z: \\storage_account_name.file.core.windows.net\YourShareName


Now you can use this share like you would any other windows file share and perform the usual file operations on it

• Using Azure Powershell

#Create a context
$ctx = New-AzureStorageContext storage_account_name storage_account_key

#Retrieve the share
$share = Get-AzureStorageShare -Context $ctx -Name YourShareName

#Create a new directory in the share
New-AzureStorageDirectory -Share $share -Path DirectoryName

# Upload a local file to the directory you just created
Set-AzureStorageFileContent -Share $share -Source C:\Temp\File.txt -Path DirectoryName

# List the contents of the directory you created
Get-AzureStorageFile -Share YourShareName -Path NewFolder

There are many more cmdlets for Azure Storage, I suggest having a look at the documentation to see what is available.

In my next post will explore the file service API.

• About
• Latest Posts

Francois Delport

I am a cloud and devops consultant, technology fan and previously a professional C# developer with a keen interest in system design and architecture. Currently I am involved in projects using Azure, the Microsoft stack and DevOps. I am based in Melbourne, Australia. Email: [email protected]

Latest posts by Francois Delport (see all)

• How To Convert A Linux Disk Or Image File To VMDK File - July 8, 2020
• Read SMART Attributes Using PowerShell And Smartmontools - February 10, 2020
• How To Install OMS Agent On Server Core - June 25, 2018

In the next few posts I am going to take Azure file service for a test drive to see what you can and cannot do with it. My aim is to have a proper exploration of the capabilities and short comings in one place instead of scouring the Internet to try and determine if you can/should use it in your projects once it goes generally available.

What is Azure file service?
Azure file service is a new capability of Azure storage to expose a file share over SMB 2.1. It is platform as a service, saving you the trouble of maintaining VMs. It is currently in preview, to enable it for your account head over to http://azure.microsoft.com/en-us/services/preview/ . You can only create shares on storage accounts created after file service was activated on your account. You will not be able to create a share on storage accounts that existed before you activated the file service preview.

What works

• Sharing files between VMs over SMB 2.1, your file share is accessible from Windows and Linux machines or any other OS implementing SMB 2.1. You can use windows file commands and API calls to access the share just like a normal windows file share.
• You get the availability, durability, scalability and redundancy of Azure storage.
• The REST API supports the usual file operations and the storage libraries in Azure SDK is a friendlier way to interact with the file share programmatically. Using the API is one way to access your file share on premise but there are others which I will look at and test later.

I think the most common use case will be sharing files between VMs and that is working at the moment. File service is cheaper than running a VM to share files as you pay per GB used, but remember this is in preview and the price will change.

What does not work (yet)

• • The SMB share cannot be mounted in Azure websites but the share can be mounted in web roles and worker roles. I will post more on this in the future and take it for a test drive myself.
  • The SMB share cannot be mounted on premise, but instead  the API can be used to access it. There is a way using WebDAV and mounting it as a local share that I will look into later on.
  • The Azure storage emulator (v3.2 at the time) does not support file service, you have to test against your storage account.
  • Active directory authentication is not supported instead access is controlled by storage keys.
  • The SMB share is only accessible by VMs in the same region.

Keep in mind file service is in preview and most of these limitations will hopefully be addressed in the future.

In my next post I will show you how to get started by creating a file share.

• About
• Latest Posts

Francois Delport

I am a cloud and devops consultant, technology fan and previously a professional C# developer with a keen interest in system design and architecture. Currently I am involved in projects using Azure, the Microsoft stack and DevOps. I am based in Melbourne, Australia. Email: [email protected]

Latest posts by Francois Delport (see all)

• How To Convert A Linux Disk Or Image File To VMDK File - July 8, 2020
• Read SMART Attributes Using PowerShell And Smartmontools - February 10, 2020
• How To Install OMS Agent On Server Core - June 25, 2018