{"schemaVersion":"jobsearcher.job.v1","id":"c8e1ab164ee7d3780847cace","url":"https://jobsearcher.com/jobs/c8e1ab164ee7d3780847cace","canonicalUrl":"https://jobsearcher.com/jobs/c8e1ab164ee7d3780847cace","title":"Senior Data Engineer – Databricks AWS","description":"*Senior Data Engineer – AWS + Databricks Lakehouse*\n\n*Healthcare / Life Sciences*\n\n*Hybrid – Indianapolis, IN *\n\nThe role involves close collaboration with data engineering teams, analytics stakeholders, and senior leadership on enterprise-wide data modernization initiatives. Requires strong expertise in Databricks Lakehouse architecture, PySpark-based engineering, cloud-native data processing, data governance, and enterprise-scale ETL/ELT modernization in regulated environments.\n\n*Key Responsibilities*\n\n_AWS + Databricks Lakehouse Engineering_\n\n* Design, develop, and optimize scalable data pipelines and Lakehouse architectures using AWS and Databricks\n* Build enterprise-grade ETL/ELT workflows for ingestion, transformation, and analytics\n* Develop scalable PySpark processing frameworks for structured and unstructured data\n* Implement Delta Lake best practices: partitioning, optimization, schema evolution, performance tuning\n* Support Medallion architecture (Bronze, Silver, Gold layers)\n\n_Data Ingestion & Processing_\n\n* Build ingestion pipelines for RWD, healthcare, operational, and analytics datasets\n* Develop reusable ingestion and transformation frameworks\n* Support batch and incremental processing workloads\n* Optimize Spark workloads, cluster utilization, and pipeline performance\n\n_AWS Cloud Integration_\n\n* Work with AWS-native services: S3, Glue, Lambda, IAM, CloudWatch, Redshift\n* Design secure, scalable integration patterns between Databricks and AWS\n* Support cloud resource optimization and cost governance\n\n_Governance, Security & Compliance_\n\n* Implement governance standards using Unity Catalog and RBAC\n* Ensure HIPAA/PHI compliance for healthcare data handling\n* Support data lineage, auditability, and access controls\n\n_Performance Optimization & Reliability_\n\n* Optimize workloads via partitioning, caching, ZORDER, cluster sizing, autoscaling\n* Improve pipeline resiliency, monitoring, alerting\n* Support CI/CD and deployment best practices\n\n_Collaboration & Leadership_\n\n* Translate business requirements into technical solutions with teams\n* Establish coding standards and reusable frameworks\n* Mentor junior engineers; facilitate documentation and knowledge transfer\n* Provide status updates to senior leadership\n\n*Required Skills & Experience*\n\n_Core Technical_\n\n* Hands-on Databricks Lakehouse platform experience\n* PySpark and Apache Spark optimization\n* Delta Lake and Medallion architecture\n* Scalable ETL/ELT framework design\n* Strong SQL and data modeling\n* Large-scale enterprise data platform experience\n\n_AWS_\n\n* AWS cloud ecosystem and data engineering services\n* S3, Glue, Lambda, IAM, Redshift, CloudWatch\n* Secure cloud architecture and integration patterns\n\n_Governance & Enterprise Architecture_\n\n* Enterprise governance and security models\n* Unity Catalog and RBAC\n* Regulated environment experience\n* Healthcare/life sciences dataset familiarity preferred\n\n*Preferred Qualifications*\n\n* Databricks Certification (Associate/Professional)\n* Enterprise modernization initiative experience\n* Healthcare/Pharma/RWD/HEOR analytics experience\n* Strong stakeholder communication and leadership\n* Experience mentoring engineering teams\n\nApplication Question(s):\n* Are you able to work in a hybrid in Indianapolis, IN?\n* How many years of hands-on experience do you have with Databricks?\n* What hands-on experience do you have with Delta Lake and Medallion architecture?\n* How many years of hands-on PySpark/Spark experience do you have, and what is the largest data volume you have worked with?\n* Have you worked with Unity Catalog, RBAC, data lineage, or governance for sensitive/regulated data?\n* Have you worked with healthcare, life sciences, pharma, RWD, HEOR, claims, EHR, or PHI data? If yes, what type of data and what was your role?\n* How many years of hands-on AWS data engineering experience do you have, and which AWS services have you worked with, such as S3, Glue, Lambda, IAM, Redshift, or CloudWatch?\n* Total work experience - This is a Senior Role, which requires minimum 10 plus years work experience. This is a W2 contract role.\n\nWork Location: Hybrid remote in Indianapolis, IN 46201","company":"Radcube","rawCompany":"radcube","city":"Indianapolis","state":"IN","isRemote":false,"isActive":false,"createdAt":"2026-09-21T10:20:20.283Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Senior Data Engineer – Databricks AWS","description":"*Senior Data Engineer – AWS + Databricks Lakehouse*\n\n*Healthcare / Life Sciences*\n\n*Hybrid – Indianapolis, IN *\n\nThe role involves close collaboration with data engineering teams, analytics stakeholders, and senior leadership on enterprise-wide data modernization initiatives. Requires strong expertise in Databricks Lakehouse architecture, PySpark-based engineering, cloud-native data processing, data governance, and enterprise-scale ETL/ELT modernization in regulated environments.\n\n*Key Responsibilities*\n\n_AWS + Databricks Lakehouse Engineering_\n\n* Design, develop, and optimize scalable data pipelines and Lakehouse architectures using AWS and Databricks\n* Build enterprise-grade ETL/ELT workflows for ingestion, transformation, and analytics\n* Develop scalable PySpark processing frameworks for structured and unstructured data\n* Implement Delta Lake best practices: partitioning, optimization, schema evolution, performance tuning\n* Support Medallion architecture (Bronze, Silver, Gold layers)\n\n_Data Ingestion & Processing_\n\n* Build ingestion pipelines for RWD, healthcare, operational, and analytics datasets\n* Develop reusable ingestion and transformation frameworks\n* Support batch and incremental processing workloads\n* Optimize Spark workloads, cluster utilization, and pipeline performance\n\n_AWS Cloud Integration_\n\n* Work with AWS-native services: S3, Glue, Lambda, IAM, CloudWatch, Redshift\n* Design secure, scalable integration patterns between Databricks and AWS\n* Support cloud resource optimization and cost governance\n\n_Governance, Security & Compliance_\n\n* Implement governance standards using Unity Catalog and RBAC\n* Ensure HIPAA/PHI compliance for healthcare data handling\n* Support data lineage, auditability, and access controls\n\n_Performance Optimization & Reliability_\n\n* Optimize workloads via partitioning, caching, ZORDER, cluster sizing, autoscaling\n* Improve pipeline resiliency, monitoring, alerting\n* Support CI/CD and deployment best practices\n\n_Collaboration & Leadership_\n\n* Translate business requirements into technical solutions with teams\n* Establish coding standards and reusable frameworks\n* Mentor junior engineers; facilitate documentation and knowledge transfer\n* Provide status updates to senior leadership\n\n*Required Skills & Experience*\n\n_Core Technical_\n\n* Hands-on Databricks Lakehouse platform experience\n* PySpark and Apache Spark optimization\n* Delta Lake and Medallion architecture\n* Scalable ETL/ELT framework design\n* Strong SQL and data modeling\n* Large-scale enterprise data platform experience\n\n_AWS_\n\n* AWS cloud ecosystem and data engineering services\n* S3, Glue, Lambda, IAM, Redshift, CloudWatch\n* Secure cloud architecture and integration patterns\n\n_Governance & Enterprise Architecture_\n\n* Enterprise governance and security models\n* Unity Catalog and RBAC\n* Regulated environment experience\n* Healthcare/life sciences dataset familiarity preferred\n\n*Preferred Qualifications*\n\n* Databricks Certification (Associate/Professional)\n* Enterprise modernization initiative experience\n* Healthcare/Pharma/RWD/HEOR analytics experience\n* Strong stakeholder communication and leadership\n* Experience mentoring engineering teams\n\nApplication Question(s):\n* Are you able to work in a hybrid in Indianapolis, IN?\n* How many years of hands-on experience do you have with Databricks?\n* What hands-on experience do you have with Delta Lake and Medallion architecture?\n* How many years of hands-on PySpark/Spark experience do you have, and what is the largest data volume you have worked with?\n* Have you worked with Unity Catalog, RBAC, data lineage, or governance for sensitive/regulated data?\n* Have you worked with healthcare, life sciences, pharma, RWD, HEOR, claims, EHR, or PHI data? If yes, what type of data and what was your role?\n* How many years of hands-on AWS data engineering experience do you have, and which AWS services have you worked with, such as S3, Glue, Lambda, IAM, Redshift, or CloudWatch?\n* Total work experience - This is a Senior Role, which requires minimum 10 plus years work experience. This is a W2 contract role.\n\nWork Location: Hybrid remote in Indianapolis, IN 46201","datePosted":"2026-09-21T10:20:20.283Z","dateModified":"2026-09-21T10:20:20.283Z","hiringOrganization":{"@type":"Organization","name":"Radcube","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Indianapolis","addressRegion":"IN","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"c8e1ab164ee7d3780847cace"},"url":"https://jobsearcher.com/jobs/c8e1ab164ee7d3780847cace"}}