Migrating Storage to Azure

Key Concepts

• Storage Migration to Azure
• Assessment of On-Premises Storage
• Azure Storage Services (Blob, Data Lake, Managed Disks, Azure Files, Azure NetApp Files, Azure Managed Lustre)
• Migration Strategies (Online, Offline, Hybrid)
• Microsoft Migration Tools (Azure File Sync, Azure Storage Mover, AzCopy)
• Storage Migration Program (SMP)
• Third-Party Migration Tools
• Modernization vs. Migration

Assessment of On-Premises Storage

The initial step involves a thorough assessment of the existing on-premises storage infrastructure. Key considerations include:

• Storage Medium: Identifying the type of storage being used (block, object, file).
  • Block: Often found in SAN environments, connected as volumes to operating systems.
  • Object: Leveraged through tools like Hadoop or S3.
  • File: Divided into NFS (historically for Linux) and SMB/CIFS (historically for Windows), used for shared file-based communication.
• Access Method: Understanding how applications interface with the storage (file shares, mounted volumes, analytics).
• Permissions: Determining the existing permissions and access control requirements. Different Azure solutions offer varying levels of granularity.
• Resilience: Evaluating the current resilience measures, such as replication within a data center or to a remote data center (e.g., SRDF).
• Capacity: Assessing the total amount of data and the potential for tiering (hot, cool, cold, archive).
• Performance Profile: Understanding the required throughput (data moved per second) and IOPS (input/output operations per second).
• Latency: Determining the acceptable latency between data request and response.
• Special Features: Identifying any specific requirements, such as parallel read/write access for high-performance computing (HPC).
• Churn: Evaluating the rate of data change (delta changes vs. complete data set changes).
• Data Interaction: Understanding the ratio of read to write operations.

Azure Storage Targets

The video outlines several Azure storage services as potential migration targets:

• Azure Blob Storage: A highly scalable object storage service.
  • Flat Namespace: Standard Blob storage without a hierarchical directory structure. Folders are part of the blob name.
  • Hierarchical Namespace (Data Lake Storage Gen2): Enables a true directory structure with POSIX-style ACLs, optimized for analytics workloads. Moving files is a metadata operation.
• Managed Disks: Block storage volumes managed by Azure, typically used with virtual machines, VM scale sets, and Azure Kubernetes Service (AKS). Sits on top of page blobs.
• Azure Files: Provides SMB and NFS file shares.
  • Supports either SMB or NFS protocol per share, not dual protocol.
• Azure NetApp Files (ANF): A managed NetApp filer service in Azure.
  • Supports SMB, NFS, and dual-protocol access (a single share can be accessed by both SMB and NFS).
  • Offers high-performance tiers and richer features compared to Azure Files.
• Azure Managed Lustre: A specialized solution for high-performance computing (HPC) scenarios.
  • Enables massive parallel read/write access to the same files.

Mapping On-Premises Storage to Azure Targets

The video provides guidance on mapping existing storage types to appropriate Azure targets:

• Block Storage (SAN): Typically migrated to Azure Managed Disks for lift-and-shift scenarios.
• Object Storage (Hadoop, S3): Generally migrated to Azure Blob Storage or Data Lake Storage Gen2, depending on analytics requirements.
• Network Attached Storage (NAS): Migrated to Azure Files or Azure NetApp Files, depending on performance, feature, and protocol requirements.

Microsoft provides a helpful decision chart in their Azure Storage documentation to guide the selection process, particularly for file-based storage. This chart considers factors like API usage, latency sensitivity, and required file protocols.

Migration Strategies

The video discusses three primary migration strategies:

• Online: Copying data over the network (internet or ExpressRoute).
• Offline: Using Azure Data Box to physically ship data to Azure.
  • Azure Data Box offers various capacities (120 TB and 525 TB).
  • Supports cross-region data restoration.
• Hybrid: Combining offline seeding with online synchronization for delta changes.

Factors influencing the choice of strategy include:

• Capacity: The amount of data to be migrated.
• Number of Files/Objects: The quantity of items to be moved.
• Bandwidth: The available network bandwidth.
• Churn: The rate of data change.

Microsoft Migration Tools

The video highlights several Microsoft tools for storage migration:

• Azure File Sync: An agent-based solution for synchronizing Windows file servers to Azure Files. Supports Windows file-based SMB going to Azure Files.
• Azure Storage Mover: An appliance-based solution for migrating file-based data to Azure.
  • Supports NFS to Blob, SMB to Azure Files.
  • Can integrate with Azure Data Box for offline seeding.
• AzCopy: A command-line utility for copying data to and from Azure Storage.
• Robocopy: A command-line directory replication command for Windows.

Storage Migration Program (SMP)

The Storage Migration Program (SMP) offers access to select ISV (Independent Software Vendor) tools at no cost, provided the migration results in net-new data being stored in Azure. Microsoft offsets the license cost of these tools.

These tools often provide richer features and capabilities compared to the native Microsoft tools, such as:

• Better upfront assessment
• Handling high churn
• More granular mapping options

Third-Party Migration Tools

In addition to the tools available through the SMP, other third-party migration tools exist. These tools may offer even more advanced features, such as:

• Advanced tiering capabilities (automatic placement of data into hot, cool, cold, or archive tiers).

While these tools require a paid license, the advanced features may result in overall cost savings by optimizing storage utilization.

Example: Comprise

The video briefly demonstrates the Comprise tool, which can be accessed for free through the SMP. Comprise uses observers (virtual machines deployed in the environment) to collect metadata about the data sources. This metadata is then used for analysis, planning, and migration. Comprise supports a wide range of storage solutions and offers advanced features such as:

• Iterative migrations
• Detailed reporting
• Customizable migration policies

Modernization Considerations

The video addresses the question of when to modernize during a migration. The general recommendation is to:

1. Migrate the data "as is" to Azure.
2. Modernize the application and data access methods once the data is in the cloud.

This approach allows for faster decommissioning of legacy infrastructure and enables the use of Azure's native tools for modernization.

Conclusion

The video provides a comprehensive overview of storage migration to Azure, covering assessment, target selection, migration strategies, and available tools. It emphasizes the importance of understanding the existing storage environment and the requirements of the applications that use the data. The video also highlights the benefits of using third-party tools, particularly through the Storage Migration Program, to address complex migration scenarios. The key takeaway is to carefully plan the migration process, considering all available options, to ensure a successful and cost-effective transition to Azure storage.