{"schemaVersion":"jobsearcher.job.v1","id":"a54377efb384188c10bf002d","url":"https://jobsearcher.com/jobs/a54377efb384188c10bf002d","canonicalUrl":"https://jobsearcher.com/jobs/a54377efb384188c10bf002d","title":"Senior Python Engineer - Databricks & Data Migration","description":"Job SummaryWe are seeking an experienced Senior Python Engineer - Databricks & Data Migration with strong hands-on expertise in Databricks, Terraform (Infrastructure as Code), and large-scale database/data platform migrations, along with familiarity with AWS cloud services.The ideal candidate should have proven experience managing Databricks environments, automating infrastructure deployments, executing complex data migrations, and optimizing data pipelines.Healthcare data experience is mandatory, including experience working with clinical, claims, patient, provider, or other healthcare datasets.This is a hands-on engineering role requiring strong technical ownership, troubleshooting capabilities, and the ability to independently drive infrastructure and migration initiatives end to end.Key ResponsibilitiesDesign, configure, deploy, and manage Databricks environments, including clusters, workflows, security, and performance optimization.Build and automate infrastructure deployments using Terraform and Infrastructure as Code (IaC).Lead end-to-end database and data platform migrations, including schema migration, ETL modernization, data validation, and performance tuning.Develop, optimize, and troubleshoot large-scale data pipelines using Databricks, PySpark, Python, and SQL.Implement scalable data ingestion, transformation, and migration workflows.Work with cloud infrastructure and services, with familiarity with AWS services such as S3, IAM, EC2, Glue, and RDS.Optimize Databricks workloads for scalability, reliability, and cost efficiency.Ensure data quality, security, and HIPAA compliance when handling healthcare datasets.Troubleshoot infrastructure, pipeline failures, data inconsistencies, and migration-related issues.Independently own technical deliverables while collaborating with engineering teams.Required Skills1. Databricks Architecture & Administration – MandatoryStrong hands-on experience with Databricks architecture, administration, and production deployments.Experience with clusters, workflows, Delta Lake, Unity Catalog, access controls, and performance optimization.Ability to troubleshoot and optimize Databricks environments.2. Terraform & Infrastructure as Code – MandatoryStrong experience with Terraform and IaC for infrastructure provisioning and automation.Experience managing infrastructure configurations, reusable modules, and deployment automation.Understanding of cloud infrastructure architecture and security.3. Database & Data Platform Migrations – MandatoryProven experience executing large-scale database or data platform migrations.Hands-on expertise in schema migration, ETL/ELT modernization, data reconciliation, validation, and performance tuning.Experience migrating legacy or on-premises data platforms to modern cloud-based architectures.4. Data Engineering – MandatoryStrong proficiency in SQL, PySpark, Python, Apache Spark, and ETL/ELT pipelines.Experience building and optimizing scalable data processing workflows.Strong troubleshooting and performance optimization capabilities.5. Healthcare Data Experience – MandatoryHands-on experience working with healthcare datasets, including clinical, claims, patient, provider, or EHR/EMR data.Understanding of HIPAA, PHI handling, healthcare data security, and compliance.Experience with HL7/FHIR healthcare interoperability standards is preferred.6. AWS Cloud Familiarity – RequiredWorking knowledge of AWS cloud infrastructure and services.Familiarity with S3, IAM, EC2, Glue, RDS, and cloud networking concepts.Understanding of cloud-based data storage, security, and integration patterns.Deep AWS architecture or administration expertise is not mandatory.Preferred QualificationsDatabricks or Terraform certifications.Experience working with Databricks on AWS, Azure, or GCP.Familiarity with AWS DMS, AWS Glue, or other migration tools.Experience with CI/CD pipelines and GitHub Actions.Experience handling large-scale healthcare data migrations.Strong communication, problem-solving, and independent technical ownership.Critical Hiring PrioritiesDatabricks Architecture & Administration – MandatoryTerraform / Infrastructure as Code – MandatoryLarge-Scale Database & Data Platform Migrations – MandatoryHealthcare Data Experience – MandatorySQL, PySpark, ETL/ELT & Performance Optimization – MandatoryAWS Cloud Familiarity – Required (Deep Expertise Not Mandatory)Independent Ownership & Troubleshooting – MandatoryImportant Note for Recruitment:We are prioritizing candidates with strong production experience in Databricks, Terraform, and large-scale data migrations, along with familiarity with AWS.Candidates with Databricks experience on Azure or GCP can also be considered, provided they have strong Terraform and migration experience and a working understanding of AWS.Profiles primarily focused on Python development, AI/ML, or LLM applications without substantial Databricks infrastructure and data migration experience will not be considered a strong fit.","company":"Gravity Infosolutions","rawCompany":"gravity infosolutions","city":"Denver","state":"CO","isRemote":false,"isActive":false,"createdAt":"2026-10-31T07:42:13.092Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Senior Python Engineer - Databricks & Data Migration","description":"Job SummaryWe are seeking an experienced Senior Python Engineer - Databricks & Data Migration with strong hands-on expertise in Databricks, Terraform (Infrastructure as Code), and large-scale database/data platform migrations, along with familiarity with AWS cloud services.The ideal candidate should have proven experience managing Databricks environments, automating infrastructure deployments, executing complex data migrations, and optimizing data pipelines.Healthcare data experience is mandatory, including experience working with clinical, claims, patient, provider, or other healthcare datasets.This is a hands-on engineering role requiring strong technical ownership, troubleshooting capabilities, and the ability to independently drive infrastructure and migration initiatives end to end.Key ResponsibilitiesDesign, configure, deploy, and manage Databricks environments, including clusters, workflows, security, and performance optimization.Build and automate infrastructure deployments using Terraform and Infrastructure as Code (IaC).Lead end-to-end database and data platform migrations, including schema migration, ETL modernization, data validation, and performance tuning.Develop, optimize, and troubleshoot large-scale data pipelines using Databricks, PySpark, Python, and SQL.Implement scalable data ingestion, transformation, and migration workflows.Work with cloud infrastructure and services, with familiarity with AWS services such as S3, IAM, EC2, Glue, and RDS.Optimize Databricks workloads for scalability, reliability, and cost efficiency.Ensure data quality, security, and HIPAA compliance when handling healthcare datasets.Troubleshoot infrastructure, pipeline failures, data inconsistencies, and migration-related issues.Independently own technical deliverables while collaborating with engineering teams.Required Skills1. Databricks Architecture & Administration – MandatoryStrong hands-on experience with Databricks architecture, administration, and production deployments.Experience with clusters, workflows, Delta Lake, Unity Catalog, access controls, and performance optimization.Ability to troubleshoot and optimize Databricks environments.2. Terraform & Infrastructure as Code – MandatoryStrong experience with Terraform and IaC for infrastructure provisioning and automation.Experience managing infrastructure configurations, reusable modules, and deployment automation.Understanding of cloud infrastructure architecture and security.3. Database & Data Platform Migrations – MandatoryProven experience executing large-scale database or data platform migrations.Hands-on expertise in schema migration, ETL/ELT modernization, data reconciliation, validation, and performance tuning.Experience migrating legacy or on-premises data platforms to modern cloud-based architectures.4. Data Engineering – MandatoryStrong proficiency in SQL, PySpark, Python, Apache Spark, and ETL/ELT pipelines.Experience building and optimizing scalable data processing workflows.Strong troubleshooting and performance optimization capabilities.5. Healthcare Data Experience – MandatoryHands-on experience working with healthcare datasets, including clinical, claims, patient, provider, or EHR/EMR data.Understanding of HIPAA, PHI handling, healthcare data security, and compliance.Experience with HL7/FHIR healthcare interoperability standards is preferred.6. AWS Cloud Familiarity – RequiredWorking knowledge of AWS cloud infrastructure and services.Familiarity with S3, IAM, EC2, Glue, RDS, and cloud networking concepts.Understanding of cloud-based data storage, security, and integration patterns.Deep AWS architecture or administration expertise is not mandatory.Preferred QualificationsDatabricks or Terraform certifications.Experience working with Databricks on AWS, Azure, or GCP.Familiarity with AWS DMS, AWS Glue, or other migration tools.Experience with CI/CD pipelines and GitHub Actions.Experience handling large-scale healthcare data migrations.Strong communication, problem-solving, and independent technical ownership.Critical Hiring PrioritiesDatabricks Architecture & Administration – MandatoryTerraform / Infrastructure as Code – MandatoryLarge-Scale Database & Data Platform Migrations – MandatoryHealthcare Data Experience – MandatorySQL, PySpark, ETL/ELT & Performance Optimization – MandatoryAWS Cloud Familiarity – Required (Deep Expertise Not Mandatory)Independent Ownership & Troubleshooting – MandatoryImportant Note for Recruitment:We are prioritizing candidates with strong production experience in Databricks, Terraform, and large-scale data migrations, along with familiarity with AWS.Candidates with Databricks experience on Azure or GCP can also be considered, provided they have strong Terraform and migration experience and a working understanding of AWS.Profiles primarily focused on Python development, AI/ML, or LLM applications without substantial Databricks infrastructure and data migration experience will not be considered a strong fit.","datePosted":"2026-10-31T07:42:13.092Z","dateModified":"2026-10-31T07:42:13.092Z","hiringOrganization":{"@type":"Organization","name":"Gravity Infosolutions","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Denver","addressRegion":"CO","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"a54377efb384188c10bf002d"},"url":"https://jobsearcher.com/jobs/a54377efb384188c10bf002d"}}