Databricks Platform Engineer
This is a hands-on engineer role covering Databricks infrastructure-as-code, deployment on AWS, and CI/CD integration with GitLab. You’ll work closely with platform engineering.
Key responsibilities:
- Deploy and manage Databricks on AWS using Terraform
- Build and maintain Terraform code for Databricks resources: workspaces, clusters, jobs, permissions and Unity Catalog
- Set up GitLab CI/CD pipelines that deploy infrastructure and Databricks assets
- Configure secure connections between GitLab, Databricks and AWSWork with platform engineers on infrastructure standards and integrations
- Must-have skills
- Terraform with the Databricks provider
- Databricks deployment on AWSGitLab CI/CD integration with Databricks
- Core AWS services: IAM, VPC, S3Good to have
- Databricks and data engineering experience
- OpenTelemetry (OTel) for observability