Skip to content

Latest commit

 

History

1 Commit

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

☁️ Azure Arc Kubernetes Provisioned Cluster Terraform Module

Registers an Azure Arc-enabled provisioned Kubernetes cluster (for example AKS on Azure Local) with a system-assigned managed identity and optional Microsoft Entra ID integration, for hashicorp/azurerm ~> 4.0.

Terraform azurerm Module Type Resources

🧩 Overview

  • Creates a single azurerm_arc_kubernetes_provisioned_cluster (the keystone this) — the Azure-side record of a provisioned Kubernetes cluster that a provisioning fabric such as Azure Local stands up and projects into Azure Resource Manager.
  • Attaches a system-assigned managed identity, so downstream services authenticate the cluster without a stored credential.
  • Exposes optional Microsoft Entra ID integration (azure_active_directory) — turn on Azure RBAC and name admin group object IDs for Entra-backed, least-privilege cluster access.
  • Controls the on-cluster Arc agent through arc_agent_auto_upgrade_enabled and an optional pinned arc_agent_desired_version.
  • Emits the cluster id, identity_principal_id, and the reported kubernetes_version / distribution; never emits a plaintext secret.

💡 Why it matters: A provisioned cluster is the Azure control-plane projection of Kubernetes that the fabric provisions (rather than a cluster you onboard after the fact). This module makes managed identity the default and keeps every authentication path credential-free, so the common call carries no stored secret.

❤️ Support this project

If this module saves you time:

🗺️ Where this fits in the family

flowchart LR
  rg["terraform-azurerm-resource-group"]
  fabric["provisioning fabric (Azure Local)"]
  entra["Microsoft Entra ID (optional)"]
  this["terraform-azurerm-arc-kubernetes-provisioned-cluster"]
  cl["azurerm_arc_kubernetes_provisioned_cluster"]
  sib["terraform-azurerm-arc-kubernetes-cluster (connected-cluster sibling)"]
  rg -->|"resource_group_name"| this
  fabric -.->|"provisions"| this
  entra -.->|"azure_active_directory"| this
  this -->|"creates"| cl
  sib -.->|"same Arc Kubernetes family"| cl
  classDef me fill:#0078D4,stroke:#004578,color:#ffffff;
  classDef target fill:#004578,stroke:#002d4d,color:#ffffff;
  classDef ext fill:#f2f2f2,stroke:#c8c8c8,color:#111111;
  class this me;
  class cl target;
  class rg,fabric,entra,sib ext;
Loading

🧬 What this module builds

flowchart TB
  in["name + resource_group_name + location + identity(SystemAssigned) + azure_active_directory?"]
  this["azurerm_arc_kubernetes_provisioned_cluster.this"]
  out["Outputs: id, identity_principal_id, kubernetes_version, distribution"]
  in --> this
  this --> out
  classDef target fill:#004578,stroke:#002d4d,color:#ffffff;
  classDef ext fill:#f2f2f2,stroke:#c8c8c8,color:#111111;
  class this target;
  class in,out ext;
Loading

Resource inventory

Resource Role Cardinality
azurerm_arc_kubernetes_provisioned_cluster keystone this single

✅ Provider / Versions

Requirement Value
Terraform >= 1.12.0
azurerm provider ~> 4.0
Provider block None — the caller configures provider "azurerm" { features {} }, auth, and subscription

Schema notes that bite (verified against the live schema):

  • Force-new: name, resource_group_name, and location are immutable — changing any replaces the cluster. Only tags (and the Arc agent / Entra settings the API allows) update in place.
  • Identity is SystemAssigned only. identity.type supports no other value — enforced here by variable validation.
  • Entra integration is optional and additive. The azure_active_directory block turns on Entra-backed auth; set azure_rbac_enabled and admin_group_object_ids for least-privilege access. Leave it null and the cluster uses no Entra integration.
  • The Arc agent tracks latest by default. Leave arc_agent_auto_upgrade_enabled null for the provider default (auto-upgrade on); pin arc_agent_desired_version only when you need a specific agent build.
  • Many attributes are computed. kubernetes_version, distribution, and the identity principal_id are reported by the provisioned cluster after creation — read them from outputs, do not set them.
  • features {} dependence. The provider will not initialize without a caller-side features {} block — expected, and owned by the root module.

🔑 Required Azure RBAC Roles / Permissions

  • Contributor on the resource group that holds the cluster.
  • The relevant Arc onboarding role for the provisioning fabric that projects the cluster into Azure.
  • Least-privilege, at the smallest scope that works.

Azure Prerequisites

  • The Microsoft.HybridContainerService / Microsoft.Kubernetes resource providers registered on the subscription.
  • The provisioning fabric (for example Azure Local) prepared to project the cluster into Azure.
  • If using Entra integration, the Entra tenant and the admin group object IDs to authorize.
  • The caller configures provider "azurerm" { features {} }, authentication, and the subscription — this module declares none of them.

📁 Module Structure

terraform-azurerm-arc-kubernetes-provisioned-cluster/
├── providers.tf   # required_version + azurerm ~> 4.0 pin; no provider block
├── variables.tf   # keystone inputs, identity, arc agent, azure_active_directory, tags + timeouts tail
├── main.tf        # azurerm_arc_kubernetes_provisioned_cluster.this
├── outputs.tf     # id first, then name, identity, kubernetes_version, distribution
├── README.md      # this document
├── SCOPE.md       # cross-module contract
├── LICENSE        # MIT
└── .gitignore

⚙️ Quick Start

The smallest real call registers the provisioned cluster with its system-assigned identity:

module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
}

ℹ️ The caller configures provider "azurerm" { features {} }, authentication, and the subscription — this module declares none of them. The cluster itself is provisioned by the fabric (for example Azure Local); this module registers and manages its Azure record.

🔌 Cross-Module Contract

Consumes

Input Type Source module
resource_group_name string terraform-azurerm-resource-group (name)
location string caller / resource group location

Emits

Output Description
id Provisioned Cluster Resource ID (first)
name Cluster name
identity_principal_id Principal ID of the system-assigned identity
kubernetes_version Reported Kubernetes version
distribution Reported Kubernetes distribution

📚 Example Library

1 · Minimal provisioned cluster
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
}

🔒 A system-assigned managed identity is attached by default — no stored credential.

2 · With tags
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
  tags = {
    environment = "production"
    owner       = "platform-team"
  }
}

ℹ️ tags update in place — name, resource_group_name, and location are force-new.

3 · Explicit system-assigned identity
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
  identity_type       = "SystemAssigned"
}

🔒 SystemAssigned is the only supported identity type; it is also the default, so this stanza only makes the choice explicit.

4 · Entra ID with Azure RBAC
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
  azure_active_directory = {
    azure_rbac_enabled     = true
    admin_group_object_ids = [var.platform_admins_group_object_id]
  }
}

🔒 Turning on azure_rbac_enabled routes cluster authorization through Azure RBAC; the named admin group gets cluster-admin access via Entra, not a static kubeconfig.

5 · Entra ID with an explicit tenant
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
  azure_active_directory = {
    azure_rbac_enabled     = true
    admin_group_object_ids = [var.platform_admins_group_object_id]
    tenant_id              = var.tenant_id
  }
}

ℹ️ Set tenant_id only when the cluster's Entra tenant differs from the provider's default tenant.

6 · Multiple admin groups
azure_active_directory = {
  azure_rbac_enabled = true
  admin_group_object_ids = [
    var.platform_admins_group_object_id,
    var.sre_oncall_group_object_id,
  ]
}

💡 List every Entra group that should hold cluster-admin; membership changes flow through Entra without a Terraform apply.

7 · Entra integration without Azure RBAC
azure_active_directory = {
  azure_rbac_enabled     = false
  admin_group_object_ids = [var.platform_admins_group_object_id]
}

ℹ️ Entra provides authentication while native Kubernetes RBAC handles authorization — use this when you manage RoleBindings in-cluster rather than through Azure RBAC.

8 · Arc agent auto-upgrade (default, made explicit)
module "arc_provisioned" {
  source                         = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                           = "arc-local-eastus"
  resource_group_name            = var.resource_group_name
  location                       = "eastus"
  arc_agent_auto_upgrade_enabled = true
}

💡 Auto-upgrade is the provider default (leave the flag null to inherit it); this stanza states the intent for readers.

9 · Pinned Arc agent version
module "arc_provisioned" {
  source                         = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                           = "arc-local-eastus"
  resource_group_name            = var.resource_group_name
  location                       = "eastus"
  arc_agent_auto_upgrade_enabled = false
  arc_agent_desired_version      = "1.18.0"
}

⚠️ Pinning arc_agent_desired_version freezes the on-cluster agent — disable auto-upgrade alongside it so the pin is not overridden, and plan the version bumps yourself.

10 · Reading identity_principal_id for an RBAC grant
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
}

module "grant" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"

  scope = var.resource_group_id
  role_assignments = {
    reader = {
      scope                = var.resource_group_id
      role_definition_name = "Reader"
      principal_id         = module.arc_provisioned.identity_principal_id
      principal_type       = "ServicePrincipal"
    }
  }
}

💡 The cluster's system-assigned identity principal is exposed as identity_principal_id for exactly this kind of least-privilege grant.

11 · Reading reported version and distribution
output "cluster_kubernetes_version" {
  value = module.arc_provisioned.kubernetes_version
}

output "cluster_distribution" {
  value = module.arc_provisioned.distribution
}

ℹ️ kubernetes_version and distribution are reported by the provisioned cluster after creation — read them, do not set them.

12 · Custom timeouts
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
  timeouts            = { create = "60m", delete = "30m" }
}

ℹ️ Provisioning a cluster through the fabric can be slow; widen create when the default is too tight.

13 · Governance tags at scale
module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = var.resource_group_name
  location            = "eastus"
  tags = {
    environment = "production"
    cost_center = "cc-4820"
    data_class  = "internal"
    managed_by  = "terraform"
  }
}

💡 tags is the one field you can freely change post-creation — use it for cost, ownership, and data-classification metadata.

14 · 🏗️ End-to-end composition

Create the resource group, register the provisioned cluster with Entra Azure RBAC, and grant its identity a role:

module "rg" {
  source   = "git::https://github.com/microsoftexpert/terraform-azurerm-resource-group.git?ref=v1.0.0"
  name     = "rg-arc-local-eastus"
  location = "eastus"
}

module "arc_provisioned" {
  source              = "git::https://github.com/microsoftexpert/terraform-azurerm-arc-kubernetes-provisioned-cluster.git?ref=v1.0.0"
  name                = "arc-local-eastus"
  resource_group_name = module.rg.name
  location            = module.rg.location
  azure_active_directory = {
    azure_rbac_enabled     = true
    admin_group_object_ids = [var.platform_admins_group_object_id]
  }
  arc_agent_auto_upgrade_enabled = true
  tags                           = { environment = "production" }
}

module "grant" {
  source = "git::https://github.com/microsoftexpert/terraform-azurerm-role-assignments.git?ref=v1.0.0"

  scope = module.rg.id
  role_assignments = {
    reader = {
      scope                = module.rg.id
      role_definition_name = "Reader"
      principal_id         = module.arc_provisioned.identity_principal_id
      principal_type       = "ServicePrincipal"
    }
  }
}

💡 The resource group feeds resource_group_name and location, Entra Azure RBAC governs cluster access, and the system-assigned identity's identity_principal_id wires straight into a least-privilege role assignment.

📥 Inputs

Required: name, resource_group_name, location. Optional: identity_type (defaults to SystemAssigned), arc_agent_auto_upgrade_enabled, arc_agent_desired_version, azure_active_directory. Universal tail: tags, timeouts.

Full schemas
name                = string # required, force-new
resource_group_name = string # required, force-new
location            = string # required, force-new

identity_type = optional(string, "SystemAssigned") # only "SystemAssigned" is supported

arc_agent_auto_upgrade_enabled = optional(bool)   # null = provider default (auto-upgrade on)
arc_agent_desired_version      = optional(string) # null = track latest

azure_active_directory = optional(object({
  admin_group_object_ids = optional(list(string))
  azure_rbac_enabled     = optional(bool)
  tenant_id              = optional(string)
})) # null = no Entra integration

tags     = optional(map(string), {})
timeouts = optional(object({ create = optional(string), read = optional(string), update = optional(string), delete = optional(string) }))

🧾 Outputs

Output Description Kind
id Resource ID of the Arc Kubernetes Provisioned Cluster (the keystone; emitted first) Passthrough
name Name of the provisioned cluster resource Passthrough
resource_group_name Name of the resource group holding the cluster resource Passthrough
location Azure region the cluster RESOURCE is registered in, as normalised by the provider Passthrough
identity_type Managed identity type attached to the cluster resource Derived
identity_principal_id Derived
identity_tenant_id Tenant ID of the cluster's system-assigned managed identity Derived
kubernetes_version Kubernetes version the cluster reports Passthrough
distribution Kubernetes distribution the cluster reports, for example "aks_management" or "k3s" Passthrough
infrastructure Infrastructure the Kubernetes cluster runs on, as reported by the Arc agents Passthrough
offering Cluster offering reported by the platform Passthrough
agent_version Passthrough
total_node_count Number of nodes the cluster reports Passthrough
total_core_count Passthrough
arc_agent_auto_upgrade_enabled Whether the Arc agents are configured to upgrade themselves automatically Passthrough
arc_agent_desired_version The pinned Arc agent version, or null when no pin is set and the agents track the current release Derived
arc_agent_version_is_pinned True when a specific Arc agent version has been requested rather than tracking the current release Derived
agent_version_pinned_with_auto_upgrade Derived
entra_integration_enabled True when an azure_active_directory block was supplied, meaning the cluster is wired to Microsoft Entra ID for authentication Derived
azure_rbac_enabled Derived
entra_tenant_id Explicit Entra tenant ID configured for cluster authentication, or null when the subscription's own tenant is used Derived
admin_group_object_ids Sorted list of Entra group object IDs granted cluster-administrator access Derived
admin_group_count Number of Entra groups granted cluster-administrator access Derived
azure_rbac_enabled_without_admin_groups Derived
entra_integration_absent Derived
tags Tags applied to the cluster resource Passthrough
manages_azure_projection_only Constant
destroy_does_not_remove_cluster_agents Constant
identity_certificate_requires_periodic_renewal Constant
shares_arm_type_with_arc_kubernetes_cluster Constant
cluster_lifecycle_is_not_governed_here Constant

🧠 Architecture Notes

  • Immutable core. name, resource_group_name, and location are all force-new. Renaming the cluster, moving its resource group, or changing its region each replaces the resource — only tags (and the Arc agent / Entra settings the API allows) update in place.
  • Managed identity only. The cluster supports SystemAssigned identity and nothing else; the module enforces this with a validation {} block rather than letting an unsupported value reach the API. The identity's principal_id is read back via identity_principal_id.
  • Entra integration is opt-in. azure_active_directory is rendered through a dynamic block that emits nothing when the variable is null, so the minimal call carries no Entra config. When present, azure_rbac_enabled plus admin_group_object_ids route authorization through Azure RBAC with an Entra-backed admin group.
  • The fabric owns provisioning. This resource is the Azure control-plane record; the underlying cluster is provisioned out of band by the fabric (for example Azure Local). Reported attributes such as kubernetes_version and distribution reflect that live cluster.
  • Arc agent lifecycle. Leave arc_agent_auto_upgrade_enabled null for the provider default; pin arc_agent_desired_version only for a deliberate freeze, and disable auto-upgrade alongside it so the pin holds.
  • No secrets. The module neither accepts nor emits a plaintext secret; authentication is identity-based throughout.
  • features {} dependence. The provider will not initialize without a caller-side features {} block — expected, and owned by the root module.

🧱 Design Principles

Concern Secure default (empty call) Opt-out (caller must type it)
Cluster identity system-assigned managed identity (no stored credential) — (SystemAssigned is the only supported value)
Entra integration none (azure_active_directory null) supply the block; set azure_rbac_enabled = true for Azure RBAC
Arc agent upgrades provider default (auto-upgrade on) pin arc_agent_desired_version and disable auto-upgrade
Secrets none accepted or emitted

🚀 Runbook

terraform init -backend=false
terraform validate
terraform fmt -check

Pin the module by immutable tag (?ref=v1.0.0), never a branch. This is plan-only; a human applies from CI.

🧪 Testing

  • terraform validate + fmt -check prove the type contract offline: the required keystone fields, the SystemAssigned-only identity validation, and the typed optional azure_active_directory object — all before any Azure call.
  • Only terraform plan against a subscription exercises the real registration: whether the resource providers are registered, whether the provisioning fabric has projected the cluster, and whether the Entra admin group object IDs resolve in the tenant.

💬 Example Output

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

Outputs:

id                    = "/subscriptions/.../resourceGroups/rg-arc-local-eastus/providers/Microsoft.HybridContainerService/provisionedClusterInstances/arc-local-eastus"
name                  = "arc-local-eastus"
identity_principal_id = "00000000-0000-0000-0000-000000000000"
kubernetes_version    = "1.28.5"
distribution          = "aks"

🔍 Troubleshooting

Symptom Cause Fix
identity_type must be "SystemAssigned" Passed another identity type Use SystemAssigned — it is the only supported value
Cluster replaced on every change Changed a force-new field (name, location, resource_group_name) Expected — those fields replace the cluster
Entra admins cannot authenticate azure_active_directory omitted or admin_group_object_ids empty Supply the block with the admin group object IDs; set azure_rbac_enabled per your authorization model
Wrong tenant on Entra login tenant_id not set and provider default differs Set azure_active_directory.tenant_id to the cluster's Entra tenant
Arc agent keeps upgrading past the pin arc_agent_auto_upgrade_enabled left on Set it to false alongside arc_agent_desired_version
RequestDisallowedByPolicy / provider not registered Microsoft.HybridContainerService / Microsoft.Kubernetes not registered Register the resource providers on the subscription
kubernetes_version / distribution empty The fabric has not finished provisioning the cluster Confirm the provisioning fabric has projected the cluster; the values populate once it reports

🔗 Related Docs

💙 "Infrastructure as Code should be standardized, consistent, and secure."