Azure Databricks Infrastructure Architect
Role: Azure Databricks Infrastructure ArchitectLocation: REMOTEInfrastructure as Code — hands-on Terraform, Bicep, or ARM Templates (not just CI/CD familiarity)Tenant-to-tenant migration — actually moving an existing Azure/Databricks environment between tenants, not general cloud migrationIdentity/platform depth — managed identities, service principals, Entra ID app registrations (distinct from Unity Catalog data-access governance)Networking — Private Endpoints, VNets, DNS in AzurePosition SummaryWe are seeking an experienced Azure Databricks Infrastructure Engineer to execute migrations of Azure Databricks environments between Azure tenants. The successful candidate will have deep expertise in Databricks, Azure cloud services, identity and access management, networking, data engineering, and platform modernization.This role is responsible for executing a migration strategy that minimizes risk, downtime, and business disruption.The ideal candidate will combine strong technical capabilities with consulting skills and be comfortable interacting with architects, platform teams, security organizations, and business stakeholders.Key ResponsibilitiesMigration Planning● Analyze tenant-specific dependencies, networking requirements, DNS, Private Endpoints, and identity integrations.● Define migration waves and cutover strategies.Databricks Migration Execution● Migrate Databricks workspaces between Azure tenants.● Rebuild or migrate:o Workspaceso Compute policieso Job scheduleso Repositorieso Workflowso Secret scopeso Access controls● Migrate notebooks, libraries, packages, and configuration artifacts.● Validate Unity Catalog and data governance configurations.● Implement infrastructure-as-code and automation where applicable.Security and Identity● Migrate and validate:o Microsoft Entra ID integrationso Managed identitieso Service principalso RBAC assignmentso Databricks groups and permissions● Coordinate Key Vault migration and secret rotation activities.● Ensure compliance with Principle of Least Privilege (PLP).● Support security reviews and governance approvals.Data Migration● Assess and migrate associated datasets and storage integrations.● Validate Delta Lake tables, metadata, and schemas.● Develop rollback and recovery procedures.Automation and DevOps● Develop Infrastructure-as-Code solutions using:o Terraformo Bicepo ARM Templates● Automate migration and validation activities wherever possible.Required Technical SkillsAzure● Azure Databricks● Microsoft Entra ID● Azure Virtual Networks● Private Endpoints● Azure Key Vault● Azure Storage Accounts● Azure Data Lake Storage Gen2● Azure Monitor● Log Analytics● Azure Policy● Azure RBACDatabricks● Workspace administration● Unity Catalog● Delta Lake● Databricks Repos● MLflow● Cluster management● Cluster policies● Secret scopes● Databricks REST APIsAutomation● Terraform● Azure DevOps● GitHub Actions● PowerShell● Azure CLI● Databricks CLIRequired Experience7+ years of Azure cloud experience.5+ years of Azure Databricks administration and engineering experience.Demonstrated experience migrating Azure resources across tenants.Experience with large-scale data platform migrations.Experience designing secure cloud landing zones.Experience implementing Infrastructure as Code.Experience working in regulated enterprise environments. Preferred QualificationsAzure Solutions Architect Expert\Databricks Administrator CertificationDatabricks Solution Architect CertificationExperience with Unity Catalog migrationsExperience with mergers, acquisitions, divestitures, or tenant-to-tenant transformationsDeliverablesThe resource should be capable of independently producing:Migration strategy and wave planDetailed migration runbooksCutover and rollback plansOperational handover documentation