Lead Databricks Engineer with Python (Only local to MN)

Dice is the leading career destination for tech experts at every stage of their careers. Our client, Sovereign Technologies, is seeking the following. Apply via Dice today!Lead Databricks Engineer (Contract-to-Hire)Location: Eagan, MN (Hybrid – 2 days/week onsite)Duration: 3–6 Months Contract-to-HireStart: ASAPJob SummaryWe are seeking a Senior/Lead Data Engineer to modernize and enhance critical enterprise data pipelines by migrating legacy SAS workflows to Python/PySpark on Databricks. This role will initially work alongside SMEs during knowledge transfer and then assume ownership of a high-impact data platform supporting healthcare analytics and reporting.Required SkillsStrong Data Engineering and application development experienceExpert-level Python development and scriptingHands-on experience with PySpark and DatabricksExperience building and maintaining large-scale distributed data pipelinesStrong SQL skills for complex data extraction, transformation, and optimizationExperience with Databricks notebooks, workflows, and cluster managementAWS experience with S3, Lambda, Glue, and EC2Experience scheduling, automating, and monitoring data pipelinesGit/version control experience in collaborative development environmentsExperience working with Agile tools, epics, and issue management< data-start=1206 data-end=1223>Nice to HaveAI/ML, Generative AI, or Automation experienceHealthcare or HEDIS data experienceLead or Principal-level Data Engineering experience< data-start=1366 data-end=1391>Key ResponsibilitiesMigrate SAS-based pipelines to Python/PySpark on DatabricksDesign, develop, and optimize scalable data processing solutionsOwn and support critical enterprise data pipelinesAutomate workflows and improve operational efficiencyCollaborate with business stakeholders, SMEs, and analytics teamsEnsure data quality, reliability, and performance across the platformMust-Have SkillsPython DevelopmentPySparkDatabricksSQLAWS (S3, Glue, Lambda, EC2)Data Pipeline DevelopmentDistributed Data ProcessingGit/Version Control