Moving From Classic (ASM) Azure to ARM May be Easier Than You Think

Some companies have been with Azure for a long time. Long enough that they still remember the (now retired) Classic portal and know just how far Azure has come. But that also leaves them in a potentially awkward position, with resources running in an environment that is not seeing any new upgrades and is slowly being absorbed into the ARM paradigm. If you fall into this category and have been thinking about how to transition your Classic resources to ARM, you are not alone.

Other than wanting to be on the New Shiny, there are some very valid reasons for considering this move. All of the new Azure services are being released in ARM only. Microsoft does a great job in making things inter-operable, but for some services, the ASM to ARM communication is just clunky. This can be especially true of the networking aspect when trying to consolidate Azure subscriptions or tenants. The most definitive reason, though, is if a company is considering using a Microsoft Cloud Solution Provider (CSP) as their Azure/Office 365/Dynamics 365 provider. CSP Azure subscriptions do not support Classic resources.

Up until recently, Microsoft did not provide a supported toolset to do this migration. The most popular and effective way to migrate was by using the migAz tool available on GitHub (https://github.com/Azure/migAz). migAz continues to evolve and provide a valuable tool set for anyone looking to move, clone or modify Azure resources and is worth any Azure admin’s time to review.

However if all you are looking for is a one-time transfer tool that will keep everything the same and not result in any duplication of resources, Microsoft may have provided the solution you need. I say “may” because there are some scenarios that the tool can’t work with, but I am getting ahead of myself.

The Move-AzureVirtualNetwork and Move-AzureService PowerShell commands can be used to migrate an entire Classic network or service to ARM. The method used is not a copy so the amount of data stored in the Classic resources has very little impact on the time it takes to transition resources. It is really based on the number of objects that need to be moved. The companion command that is required to completely move IaaS resource to ARM is the Move-AzureStorageAccount command.

Before we get into how the commands work, it is worthwhile to understand how Azure views VMs. We focus on VMs (and IaaS resources like networks) when we talk about moving because PaaS and SaaS services don’t need to be migrated, they exist outside of the IaaS infrastructure already. A VM, in Azure terms, is really a set of metadata that describes how to provision the resources required by a VM when it is started. What kind of network card and IPs does it have? How many and where are the disks? You get the point. This is why when a machine is shut down (deallocated), there is no cost for the VM, it technically doesn’t exist. Sure, the disks are there with all the storage, so you still pay storage, but the VM itself is nothing more than a template.

Changing that template is a relatively simple process. Anyone who has worked with availability sets in ARM already knows the pain of having to delete and recreate a VM to add/change/remove a VM/availability set combination. Deleting the VM removes the template file, but all of the resources remain behind. The NIC, NSG, disks, all of it. So recreating a VM is really creating a new template and pointing it at the existing resources.

So how does this help in the Move commands? Microsoft takes advantage of the nature of the template by simply creating the new ARM metadata from the ASM metadata. Both templates (the ASM and ARM files) reference the same resources. So during the actual move (more on the process in a bit) the VMs and disks are being referenced by 2 instances. Long term this is not sustainable, but for the duration of the move, its magic. I ran a test on the migration and was able to stay logged in and executing apps on a VM while it moved. There was no downtime. I am not recommending this approach at all for production moves, but it highlights how the change has nothing to do with the resources and that it is all done with the metadata.

The commands themselves have three steps to them: Validation, Prepare and Commit. They are self-explanatory, but are still worth looking at deeper.

The validation step will review all of the resources in the ASM network or service looking for unsupported configurations. Things like ACLs on VM endpoints and multiple availability sets on a load balancer. These configurations are valid and supported in ASM, but because of how the ARM resources are used there is not a simple, automated way to convert them. Human intervention is required. This doesn’t mean that the Move commands can’t be used, it just means that there may be additional scripts that need to be run before and/or after the move to reach the desired state. In many cases this translation of services would need to be done even if the migration was performed with the migAz tool.

Once the validation passes, you can run the prepare. This step is where the duplication of the metadata happens. If you are watching in the portal you will see the duplication as a VM/storage account with the same name will show up as a Classic and ARM instance. Even though the validation was successful it is still possible to have this step fail. Most of the time it happens because of a lack of available ARM resources (scripts on how to check this are below). If the prepare does fail, there is an abort option that will reverse was has been prepped, removing the duplicate metadata. It is really important to understand that while VM’s and Storage accounts are in the prepared state, nothing can be done to them. They can’t even be started or stopped. This is not a place you want to be for more than a few hours but you do want to take the time to validate what has been created. Once the commit is run there is no going back. When you are comfortable with the new objects, continue with the Commit. If it doesn’t look right, run an Abort. You can prepare and abort as often as needed until everything looks right.

The commit is the simplest stage. It cleans up all the ASM resources. There is no abort option here. There is also no undo. Once the commit is run the only way to get VMs back into Classic is to recover them from a backup. So make sure you have backups.

Once the network or service move has completed, the process must be repeated for storage accounts. The validation, prepare and commit must be run after the network or service move because the storage validation will fail if there are Classic VMs with disks in the account. Note that managed disks do not live in storage accounts and will need to be moved separately using the AzCopy command. The timing on the storage move is not critical, though it should be soon as soon as possible after the VM moves. Once the VMs are moved over they can run for a while with stargaze accounts still in Classic. This makes it easier to deal with any issues that may come up with the storage validation. There is no need to cancel the move and restore back to Classic unless something comes up that cannot be easily fixed.

Getting back to the backups, there are things that will not get moved using these tools. Like backups. Once the dust has settled, make sure to go back and manually remove or migrate anything left behind. There is no reason to pay for resources that can’t be used.

So far there has been a lot of theory, so now on to the scripting. Below are the minimum scripts that should be run during a migration using these commands. I also have links to the Microsoft pages for the actual move scripts for all the additional details. Keep in mind that additional scripts may be needed to prepare/cleanup resources that can’t be moved with the Move commands.

It should go without saying, but as there is no undo I will be very specific: this needs to be tested thoroughly before being run on an actual production environment. DEV/TEST environments are great for this, but if you don’t have them, setting up a simple environment with a few small VMs, a network and any VPN, Load Balancer, etc. will be well worth the effort.

Step 1 – List out ASM resources

This script is a little clunky as there is no single script to enumerate ASM VMs. Use this information to compare against available ARM resources to make sure you are covered. Note, the output goes to a CSV file that can be opened in Excel and then used in a pivot table.

$filename = "c:\migrate\cores.csv"
"VM Size, Core Count" | out-file -FilePath $filename -Encoding ascii
$vms = get-azurevm
foreach($vm in $vms)
{
    "$($vm.InstanceSize), `
        $((get-azurerolesize -InstanceSize $vm.InstanceSize).Cores)" | `
        out-file -FilePath $filename -Append -Encoding ascii
}

The output in Excel with an added pivot table:

Spreadsheet

Step 2 – List available ARM resources

Listing ARM resources is a lot easier, it can be done with a provided script. Note that if a specific type of resource does not show, it means that the default limits are in place and will likely need to be increased. Additional resources can be added by creating a support request with Microsoft. A list of default limits can be found at https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits.

To list the VM resources available, run this command for each region being migrated:

Get-AzureRmVMUsage -Location ‘East US’ | `
    select @{label="Name";expression={$_.name.LocalizedValue}}, `
    currentvalue,limit | Format-Table -AutoSize 
Name                             CurrentValue Limit
----                             ------------ -----
Availability Sets                           8  2000
Total Regional vCPUs                       19    30
Virtual Machines                           12 10000
Virtual Machine Scale Sets                  0  2000
Standard A0-A7 Family vCPUs                11    30
Standard DSv2 Family vCPUs                  8    10
Basic A Family vCPUs                        0    10
Standard A8-A11 Family vCPUs                0    10
Standard D Family vCPUs                     0    10

 

To list network resources available run this command for each region being migrated:

Get-AzureRmNetworkUsage -Location 'EastUS' | `
    Select ResourceType, CurrentValue, Limit | Format-Table -AutoSize 
ResourceType                                    CurrentValue      Limit
------------                                    ------------      -----
Virtual Networks                                           2         50
Static Public IP Addresses                                 0         20
Network Security Groups                                    2        100
Public IP Addresses                                       13         60
Public Ip Prefixes                                         0 2147483647
Network Interfaces                                        13      24000
Load Balancers                                             4        100
Application Gateways                                       0         50

 

Compare the lists to see what needs to be increased. If you are concerned about resources other than cores, like public IPs or NSGs, similar ASM scripts can be written for them.

Step 3 – Prepare the subscription for upgrade

The subscription has to be told there will be an upgrade done on it. Without this step the subscription will not support duplicate metadata definitions. This script only needs to be run once per subscription.

Register-AzureRmResourceProvider -ProviderNamespace Microsoft.ClassicInfrastructureMigrate

If you are not sure if this has been run, there is no harm in running it multiple times, however you can also check if the registration has been done with this:

Get-AzureRmResourceProvider -ProviderNamespace Microsoft.ClassicInfrastructureMigrate

 

Step 4 – Run the Validation

For network moves, which is all I will cover as they are by far more common than service moves, simply because of how ASM deploys resources, run the following:

$validate = Move-AzureVirtualNetwork -Validate -VirtualNetworkName $vnet

The $validate variable will hold the returned success and failure messages.

Note that the vnet name may not be what it shows in the portal. Get the actual vnet name by running

Get-AzureVnetSite | Select -Property Name

 

Step 5 – Run the Prepare

Assuming the validation completes and you are ready to move forward, run the prepare. This is generally the longest step in the process.

Move-AzureVirtualNetwork -Prepare -VirtualNetworkName $vnet

Note if the prepare fails, run an abort to return to a sustainable state.

Move-AzureVirtualNetwork -Abort -VirtualNetworkName $vnet

Step 6 – Run the Commit

After validating that the prepared resources look good, the last step is to commit the move. Remember, there is no undo.

Move-AzureVirtualNetwork -Commit -VirtualNetworkName $vnet

 

Step 7 – Move the Storage Accounts

Similar to the network or service, storage accounts needed to be validated, prepared and committed. Make sure to run this set of command for each storage account that is needed and that anything that will be left behind does not have resources in these storage accounts! Remember, there is no undo. Note that managed disks do not live in a storage account and cannot be moved with this set of tools. AZCopy or the Azure Storage Explorer are needed to move managed disks.

The list of commands are basically the same:

$validate = Move-AzureStorageAccount -Validate -StorageAccountName $storageAccountName

Move-AzureStorageAccount -Prepare -StorageAccountName $storageAccountName

Move-AzureStorageAccount -Abort -StorageAccountName $storageAccountName

Move-AzureStorageAccount -Commit -StorageAccountName $storageAccountName

 

Step 8 – Clean up and go home

Part of the cleanup should be to move resources to the correct resource group. When the move happens, resources will be moved to a resource group named after the service they were in with a -migrate appended to it. The vnet will get it own resource group. Consolidating these after the fact will make management easier going forward. This can be done either through the portal or via PowerShell.

 

When considering moving resources to ARM, the complexity of the environment as well as the desired end state will drive the tool selection. migAz is really flexible but may be overkill for simpler environments or in cases where duplication of resources is an issue. Understanding how Azure works with resources and the limitations of the tools will go a long way to facilitating a quicker conversion.

Have you found any other useful tools or written some scripts to help out with resource management? Let me know!

 

Links to Microsoft resources:

Move Azure virtual network:

https://docs.microsoft.com/en-us/powershell/module/azure/move-azurevirtualnetwork?view=azuresmps-4.0.0

 

Move Azure Service:

https://docs.microsoft.com/en-us/powershell/module/azure/move-azureservice?view=azuresmps-4.0.0

 

Move Azure storage account:

https://docs.microsoft.com/en-us/powershell/module/azure/move-azurestorageaccount?view=azuresmps-4.0.0

 

Azure service limits:

https://docs.microsoft.com/en-us/azure/azure-subscription-service-limits.

 

AZCopy:

https://docs.microsoft.com/en-us/azure/storage/storage-use-azcopy

 

Azure Storage Explorer:

https://azure.microsoft.com/en-us/features/storage-explorer/

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s