Prerequisites

Microsoft Azure Resource Providers

The following Microsoft Azure resource providers are registered.

• Microsoft.ContainerService
• Microsoft.Network
• Microsoft.Compute
• Microsoft.Storage
• Microsoft.KeyVault
• Microsoft.ManagedIdentity

Creating a Storage Account in Azure

An Azure Storage Account is required to store cluster backup data. The following steps guide you through creating the storage account, configuring the required blob container, provisioning a Key Vault, and assigning the necessary permissions.

1. Creating the Storage Account

Create an Azure Storage Account to serve as the backend for cluster backup and restore operations using Velero.

To create a storage account, perform the following steps:

1. Sign in to the Azure portal.
2. Navigate to Home > Storage accounts, then select Create.
3. On the Basics tab, provide the required details:
   • Subscription: The Azure subscription to bill for this resource.
   • Resource group: The logical container that groups related Azure resources.
   • Storage account name: A globally unique name using only lowercase letters and numbers.
   • Region: The Azure datacenter location where the storage account will be hosted.
   • Primary service: The primary Azure service that will use this storage account (e.g., Blob Storage, Azure Files).
   • Performance tier: Choose Standard for general-purpose workloads or Premium for low-latency requirements.
   • Redundancy option: The replication strategy for your data (e.g., LRS, GRS, ZRS).
4. Keep the default values for the remaining tabs unless a specific project requirement applies.
5. On the Review + create tab, review the configuration, then select Create.
6. After deployment completes, select Go to resource to open the newly created storage account.

Note: A blob container named velero must be created inside the storage account before installation. If the container does not exist, the installation will fail when the deployment attempts to access it.

2. Creating a container inside the storage account

Create a blob container named velero within the storage account. This container is used by Velero to store backup snapshots.

To create a container inside the storage account, perform the following steps:

1. Navigate to Home > Storage accounts, then select the required storage account.
2. In the left pane, under Data storage, select Containers.
3. Select Add Container.
4. In the New container pane, enter velero as the container name.
5. Set the anonymous access level to Private (no anonymous access).
6. Select Create, and verify that the velero container appears in the container list.

3. Creating a Key Vault

Create an Azure Key Vault to securely store the encryption keys used during cluster deployment.

To create a Key Vault, perform the following steps:

1. Navigate to Home > Key vaults, then select Create.
2. Fill out the required details in the Basics section.
3. Leave the Access configuration, Networking, and Tags sections at their default values.
4. Select Review + create, then select Create.

4. Assigning the Key Vault Administrator role

The Key Vault Administrator role must be assigned to the identity performing the installation. This role is required to create, update, and manage secrets, certificates, and keys within the Azure Key Vault used during deployment. Without this permission, the installation may fail.

Verifying current role assignments

To verify current role assignments, perform the following steps:

1. Navigate to Home > Key vaults, then select the key vault.
2. In the left pane, select Access control (IAM), then select View my access.
3. Confirm whether the Key Vault Administrator role is already assigned.

Adding the Key Vault Administrator role

To add the Key Vault Administrator role, perform the following steps:

1. In Access control (IAM), select Add role assignment.
2. Search and select Key Vault Administrator.
3. Under Members, add the required members.
4. Leave Conditions unchanged, then select Review + assign.
5. Verify that the Key Vault Administrator role is now listed for the selected key vault.

5. Adding Keys to the Key Vault

Generate an encryption key inside the Key Vault. This key is referenced during cluster installation to encrypt storage and other sensitive resources.

To add keys to the Key Vault, perform the following steps:

1. Navigate to Home > Key vaults, then select the key vault.
2. In the left pane, select Objects > Keys, then select Generate/Import.
3. Provide the required key details and complete key generation.

Note the values for the following. They are required during cluster installation.

• Storage account name
• Key vault
• Key name

Permissions to deploy AKS

Update the following custom role permissions.

Before you begin

• Identify the resource group where the AKS cluster will be created.
• Ensure that a custom role is already created in the subscription or resource group.
• Complete this configuration before starting the AKS installation process.

To update the permission for the custom role, perform the following steps:

1. Navigate to the required resource group in the Azure portal.

2. Open Access control (IAM) for the selected resource group.

3. Locate the existing custom role.

4. Select the custom role and click Edit.

5. Select JSON and click Edit.

   Update the role by adding the following permissions:

  {
    "id": "/subscriptions/<SUBSCRIPTION-ID>/providers/Microsoft.Authorization/roleDefinitions/<ROLE-DEFINITION-ID>",
    "properties": {
        "roleName": "<ROLE-NAME>",
        "description": "",
        "assignableScopes": [
            "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RG-Name>"
        ],
        "permissions": [
            {
                "Actions": [
                    "Microsoft.Authorization/roleAssignments/read",
                    "Microsoft.Authorization/roleAssignments/write",
                    "Microsoft.Authorization/roleDefinitions/read",
                    "Microsoft.Authorization/roleDefinitions/write",
                    "Microsoft.ContainerService/managedClusters/read",
                    "Microsoft.ContainerService/managedClusters/write",
                    "Microsoft.KeyVault/vaults/read",
                    "Microsoft.KeyVault/vaults/write",
                    "Microsoft.ManagedIdentity/userAssignedIdentities/assign/action",
                    "Microsoft.ManagedIdentity/userAssignedIdentities/read",
                    "Microsoft.Network/virtualNetworks/read",
                    "Microsoft.Network/virtualNetworks/subnets/join/action",
                    "Microsoft.Network/virtualNetworks/subnets/read",
                    "Microsoft.Resources/subscriptions/resourceGroups/read",
                    "Microsoft.Storage/locations/checknameavailability/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/write",
                    "Microsoft.Storage/storageAccounts/blobServices/read",
                    "Microsoft.Storage/storageAccounts/blobServices/write",
                    "Microsoft.Storage/storageAccounts/fileServices/read",
                    "Microsoft.Storage/storageAccounts/fileServices/write",
                    "Microsoft.Storage/storageAccounts/listKeys/action",
                    "Microsoft.Storage/storageAccounts/read",
                    "Microsoft.Storage/storageAccounts/write",
                    "Microsoft.Network/privateDnsZones/read",
                    "Microsoft.Network/privateDnsZones/A/read",
                    "Microsoft.Network/privateDnsZones/A/write",
                    "Microsoft.Network/privateDnsZones/write",
                    "Microsoft.Network/privateDnsZones/join/action",
                    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/read",
                    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/write",
                    "Microsoft.ContainerService/managedClusters/listClusterUserCredential/action",
                    "Microsoft.ContainerService/managedClusters/agentPools/read",
                    "Microsoft.ContainerService/managedClusters/agentPools/write",
                    "Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials/write",
                    "Microsoft.KeyVault/vaults/read"
                ],
                "notActions": [],
                "dataActions": [],
                "notDataActions": []
            }
        ]
    }
}

6. Click Save after updating the permissions.
7. Click Review + update to finalise the changes.

Jump Box or Local Machine

A jump box serves as a secure, controlled access point for executing installation commands and managing cluster resources. It ensures that all operations originate from a trusted environment with direct network access to the PPC cluster, avoiding connectivity issues that may arise from local machines with restricted network paths or inconsistent tooling.

• Use a dedicated Debian or Red Hat jump box created in Microsoft Azure. Do not use a jump box hosted on any other cloud.

• The jump box must be provisioned in the same Azure region where the PPC cluster will be deployed to ensure proper connectivity and access to regional resources.

  For example, if the PPC cluster is deployed in eastus, the jump box must also be created in eastus. Placing the jump box in a different region, such as westeurope, may result in network access failures or latency issues when connecting to cluster endpoints.

Microsoft Azure Region

For automated installation

Identify the Azure region where the cluster is to be deployed.

The deployment region is automatically detected from the resource group’s location during the bootstrap process. Ensure that the resource group is created in the intended deployment region before running the installer.

All resources must be in the same region.
For more information about Microsoft Azure resources, refer to Microsoft Azure Resource IDs from Infrastructure Team.

For component-based installation

Identify the Azure region where the cluster is to be deployed.

Region must be specified as azure_location in the terraform.tfvars file during the 00_backup_storage and 02_aks_cluster stages.

All resources must be in the same region.
For more information about Microsoft Azure resources, refer to Microsoft Azure Resource IDs from Infrastructure Team.

Microsoft Azure Resource IDs from Infrastructure Team

Deployment Resource Group

Before proceeding with the installation, identify the Azure Resource Group where the cluster is to be deployed.

• For automated deployment, the value of this resource group name must be specified during the input prompt while running the ./bootstrap.sh script.
• For component-based deployment, the value of this resource group name must be specified in the resource_group_name field in the terraform.tfvars files, wherever required.

The following resources are provisioned within the resource group specified during deployment.

Pre-existing Resources

The following resources must be provisioned by the IT team before installation.

They can reside in any resource group in the same region. The installer references each resource by its entire Azure resource ID.

Ensure the deploying identity, such as, service principal or managed identity, has the appropriate permissions on these resources.

AKS User-Assigned Managed Identity Permissions

The AKS User-Assigned Managed Identity (id-aks-applianceframework) must be granted a custom role with the following permissions on the resource group containing the Private DNS Zone and Virtual Network. These permissions enable AKS to manage private DNS records and join virtual networks during cluster operations.

To configure User-Assigned Managed Identity with a custom Azure role containing the permissions required for AKS deployment and management operations within the specified resource group, perform the following steps:

1. Creating a Custom Role

1. In the Azure portal, navigate to Resource Groups.
2. Select the resource group that contains the AKS resources.
3. Open Access Control (IAM).
4. Select Add > Add custom role.
5. Provide a name and description for the custom role.
6. Select JSON and add the following permissions.

{
  "Name": "<ROLE-NAME>",
  "Description": "",
  "AssignableScopes": [
    "/subscriptions/<SUBSCRIPTION-ID>/resourceGroups/<RESOURCE-GROUP>"
  ],
  "Actions": [
    "Microsoft.Network/privateDnsZones/read",
    "Microsoft.Network/privateDnsZones/A/read",
    "Microsoft.Network/privateDnsZones/A/write",
    "Microsoft.Network/privateDnsZones/A/delete",
    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/read",
    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/write",
    "Microsoft.Network/privateDnsZones/virtualNetworkLinks/delete",
    "Microsoft.Network/virtualNetworks/subnets/read",
    "Microsoft.Network/virtualNetworks/subnets/join/action",
    "Microsoft.Network/privateDnsZones/write",
    "Microsoft.Network/virtualNetworks/join/action"
  ],
  "NotActions": [],
  "DataActions": [],
  "NotDataActions": []
}

7. Review the permissions and create the custom role.

Note: These permissions are required when using a private AKS cluster with a pre-existing Private DNS Zone (private_dns_zone_id). Without them, cluster provisioning and DNS resolution will fail.

2. Assigning the custom role to the Managed Identity

1. On the Azure portal, navigate to Managed Identities.

2. Select Create.

3. Specify the following:

   • Subscription
   • Resource group
   • Managed identity name
   • Region

4. Select Review + Create and then Create.

   Wait for the User-Assigned Managed Identity (UAMI) to be provisioned.

5. Open the newly created User-Assigned Managed Identity.

6. Select Azure role assignments.

7. Select Add role assignment.

8. Set the scope to the target resource group.

9. Select the custom role created in the previous section.

10. Assign the role to the User-Assigned Managed Identity.

11. Verify that the role assignment appears under the managed identity’s role assignments.

Zone Requirements

The cluster requires a minimum of two availability zones for high availability. Three-zone cluster deployments are supported and recommended for production.

Ensure the following requirements are met before deploying:

The bootstrap script does not prompt for zone selection, it uses the default three zone configuration. The default zone configuration is ["1", "2", "3"]. Not all Azure regions support three availability zones, and not all VM SKUs are available in every zone. Using a zone that does not support your VM SKU causes cluster creation to fail.

To verify the zone that is supported in the required region for PPC deployment, run the following command.

# List availability zones for your region
az account list-locations \
  --query "[?name=='<region>'].availabilityZoneMappings[].logicalZone" \
  -o table

Replace <region> with your Azure region, for example eastus.

Perform the following steps to modify the number of zones

1. Navigate to iac/azure/cluster/modules.

2. Enter the value of the supported zones in both terraform.tfvars files for 02_aks_cluster and 03_node_pool modules.

   # 02_aks_cluster/terraform.tfvars
   system_node_pool_zones = ["<value of supported zone 1>", "<value of supported zone 2>"]    # adjust to the zones your region supports
   
   # 03_node_pool/terraform.tfvars
   node_pool_zones = ["<value of supported zone 1>", "<value of supported zone 2>"]           # must match system_node_pool_zones exactly
   

   where <value of supported zone 1> and <value of supported zone 2> are the supported zones in the required Azure region.

   For more information on editing the 02_aks_cluster, and 03_node_pool modules, refer to Manual component-based installation.

Node pool sizing

The step [1b] Select Node Architecture in the bootstrap script prompts to select the node architecture interactively during deployment.

For arm64 and amd64, the following are the default system and user node pool configurations.

amd64 (x86-64)

arm64

Before creating the cluster, verify that your target Azure region provides the required availability zones and that your subscription can launch the required VM sizes in those zones.

# System node pool SKU
az vm list-skus \
  --subscription <subscription ID> \
  --location <region>\
  --size <VM Size>\
  --output table

where

• subscription ID is the subscription ID for the Microsoft Azure account.
• <region> is the name of the region where PPC is to be deployed.
• <VM Size> is one of the supported amd64 or arm64 VM sizes listed above. Run the command once for each size. Ensure that the access to launch both, system and user node pool, are available.

A sample output is as follows:

ResourceType     Locations    Name              Zones    Restrictions
---------------  -----------  ----------------  -------  --------------
virtualMachines  eastus       Standard_D8as_v5  1,2,3    NotAvailableForSubscription type: Zone, locations: eastus zones: 3,1

where,

• Zones are the availability zones in which the VM size can be deployed. At least two zones must be available. If the Zones column is empty, then the region does not offer availability zones for that size. In such scenarios, select a different region.
• Restrictions If the value is None, no action is required. If any restriction is listed, take the relevant action, such as, choose a different region, or request access, increase quota, before proceeding.

AKS cluster tier

The sku_tier setting controls the AKS API server uptime SLA, available node capacity, and supported features.

Set sku_tier = "Standard" for all production deployments. Use "Premium" if your organization requires long-term support (LTS) guarantees.

In 02_aks_cluster/terraform.tfvars, edit the following value:

sku_tier = "Standard"

For more information on editing the 02_aks_cluster module, refer to Manual component-based installation.