{"schemaVersion":"jobsearcher.job.v1","id":"bac9abcaaf30619b2c232dde","url":"https://jobsearcher.com/jobs/bac9abcaaf30619b2c232dde","canonicalUrl":"https://jobsearcher.com/jobs/bac9abcaaf30619b2c232dde","title":"Lead Databricks Platform Engineer (Remote US)","description":"OverviewAs a Senior Databricks Platform Engineer, you will design, secure, and operate governed Azure Databricks Lakehouse capabilities for a cloud-based analytics platform supporting government data initiatives. You'll collaborate with cross-functional teams to move data products from ingestion to analysis-ready datasets and production operations. You'll implement governance, automation, and scalable data pipelines to enable enterprise AI and analytics. This role offers impactful work in a mission-driven government context with modern data tools and CI/CD practices.ResponsibilitiesDesign, configure, and operate Azure Databricks workspaces, clusters, jobs, workflows, catalogs, storage credentials, and service principalsBuild and optimize batch and real-time ingestion and transformation pipelines using Python, PySpark, SQL, Delta Lake, Delta Live Tables, Auto Loader, and Apache SparkImplement medallion architecture, dimensional models, CDC, schema evolution, data quality checks, retry/rollback, and reproducible processing patternsConfigure Unity Catalog governance (RBAC, lineage, retention, provisioning) and secure integration with ADLS Gen2 and other storageDevelop interfaces for AI/analytics teams to access governed training, inference, and reporting data across initiativesAutomate Databricks deployments and environment configuration using Terraform, Databricks Asset Bundles, GitHub Actions, Azure DevOps, or similar toolsIntegrate Databricks with Azure Data Factory, Kafka, Airflow, APIs, Snowflake, Synapse, and other enterprise data servicesMonitor pipeline, cluster, and workspace health; diagnose performance and data issues using Azure Monitor, Databricks logs, SplunkSupport security, ATO, vulnerability remediation, business continuity, disaster recovery, and operational readiness for Databricks environmentsParticipate in Agile planning, code reviews, demos, retrospectives; document architecture, runbooks, data models, governance decisions, and deployment procedures in version-controlled reposKey requirementsBA/BS/MS in Computer Science, Engineering, Data Science, or equivalent experience5+ years of data engineering experience3+ years supporting Azure Databricks and related Azure data servicesExpertise in Apache Spark, Delta Lake, Python, SQL, and ScalaExperience with Azure Data Factory, Data Lake Storage, Synapse, Databricks workflows, data modeling, and big data technologiesExperience configuring and managing Databricks workspaces, clusters, security controls, access management, and cost optimizationExperience implementing CI/CD or IaC for data platforms using Git, Terraform, Databricks Asset Bundles, GitHub Actions, Azure DevOpsExperience with Agile delivery, data governance, data security, performance troubleshooting, and stakeholder communicationAbility to satisfy FDIC eligibility and U.S. citizenship requirementsU.S. Citizenship required; candidate must reside and work in the U.S.Effective communicator with technical and business stakeholdersCross-functional collaboration across data engineering, AI, cloud, security, and operationsStructured problem solving and issue diagnosisAzure DatabricksDelta LakeDelta Live Tables","company":"ICF","rawCompany":"icf","city":"Bloomington","state":"IL","isRemote":true,"isActive":false,"createdAt":"2026-10-26T13:27:43.392Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Lead Databricks Platform Engineer (Remote US)","description":"OverviewAs a Senior Databricks Platform Engineer, you will design, secure, and operate governed Azure Databricks Lakehouse capabilities for a cloud-based analytics platform supporting government data initiatives. You'll collaborate with cross-functional teams to move data products from ingestion to analysis-ready datasets and production operations. You'll implement governance, automation, and scalable data pipelines to enable enterprise AI and analytics. This role offers impactful work in a mission-driven government context with modern data tools and CI/CD practices.ResponsibilitiesDesign, configure, and operate Azure Databricks workspaces, clusters, jobs, workflows, catalogs, storage credentials, and service principalsBuild and optimize batch and real-time ingestion and transformation pipelines using Python, PySpark, SQL, Delta Lake, Delta Live Tables, Auto Loader, and Apache SparkImplement medallion architecture, dimensional models, CDC, schema evolution, data quality checks, retry/rollback, and reproducible processing patternsConfigure Unity Catalog governance (RBAC, lineage, retention, provisioning) and secure integration with ADLS Gen2 and other storageDevelop interfaces for AI/analytics teams to access governed training, inference, and reporting data across initiativesAutomate Databricks deployments and environment configuration using Terraform, Databricks Asset Bundles, GitHub Actions, Azure DevOps, or similar toolsIntegrate Databricks with Azure Data Factory, Kafka, Airflow, APIs, Snowflake, Synapse, and other enterprise data servicesMonitor pipeline, cluster, and workspace health; diagnose performance and data issues using Azure Monitor, Databricks logs, SplunkSupport security, ATO, vulnerability remediation, business continuity, disaster recovery, and operational readiness for Databricks environmentsParticipate in Agile planning, code reviews, demos, retrospectives; document architecture, runbooks, data models, governance decisions, and deployment procedures in version-controlled reposKey requirementsBA/BS/MS in Computer Science, Engineering, Data Science, or equivalent experience5+ years of data engineering experience3+ years supporting Azure Databricks and related Azure data servicesExpertise in Apache Spark, Delta Lake, Python, SQL, and ScalaExperience with Azure Data Factory, Data Lake Storage, Synapse, Databricks workflows, data modeling, and big data technologiesExperience configuring and managing Databricks workspaces, clusters, security controls, access management, and cost optimizationExperience implementing CI/CD or IaC for data platforms using Git, Terraform, Databricks Asset Bundles, GitHub Actions, Azure DevOpsExperience with Agile delivery, data governance, data security, performance troubleshooting, and stakeholder communicationAbility to satisfy FDIC eligibility and U.S. citizenship requirementsU.S. Citizenship required; candidate must reside and work in the U.S.Effective communicator with technical and business stakeholdersCross-functional collaboration across data engineering, AI, cloud, security, and operationsStructured problem solving and issue diagnosisAzure DatabricksDelta LakeDelta Live Tables","datePosted":"2026-10-26T13:27:43.392Z","dateModified":"2026-10-26T13:27:43.392Z","hiringOrganization":{"@type":"Organization","name":"ICF","sameAs":"https://jobsearcher.com"},"jobLocationType":"TELECOMMUTE","applicantLocationRequirements":{"@type":"Country","name":"US"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Bloomington","addressRegion":"IL","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"bac9abcaaf30619b2c232dde"},"url":"https://jobsearcher.com/jobs/bac9abcaaf30619b2c232dde"}}