Senior Data Engineer

Senior Data EngineerJoin our AI Data Products Scrum Team as a Senior Data Engineer, supporting a Fortune 50 healthcare insurance organization in a fully remote capacity. In this hands-on role, you will be instrumental in building, optimizing, and operationalizing scalable data pipelines and reusable data assets that power advanced analytics, machine learning, and generative AI initiatives. You will work within a modern distributed data environment, collaborating with cross-functional teams to deliver reliable, production-ready healthcare data solutions. This is an opportunity to make a significant impact in a dynamic, innovative environment while enjoying comprehensive benefits and competitive compensation.Remote RoleContract to HireCompensation: $80 - $90/hour, DOEDesign, build, and maintain scalable batch and streaming data pipelines using modern distributed data technologiesDevelop production data solutions using SQL, Python, and PySpark, selecting the appropriate tool for each workloadTroubleshoot, optimize, and enhance existing data pipelines to meet new data requirements and improve performanceCreate reusable ingestion, transformation, reconciliation, and data-product patterns for analytics and AI use casesImplement data-quality checks, monitoring, lineage, documentation, and provide reliable operational support for production pipelinesUtilize Git and CI/CD practices to test, review, version, deploy, and promote data-engineering assets across environmentsCollaborate with lead engineers, application teams, data scientists, and platform teams on production healthcare solutions, market expansion, and AI-assisted workflowsSupport AI-enabling data assets such as feature tables, vector indices, embeddings, or semantic contextEnsure compliance with data governance, cataloging, lineage, and discoverability practices, especially in regulated environmentsSenior-level, hands-on experience designing, building, and supporting production data pipelines in a modern distributed data environmentStrong production experience with Databricks Data Engineering, Spark distributed computing, or the Hadoop frameworkStrong SQL skills and practical experience using Python and/or PySpark for production data-engineering workHands-on experience with DevOps, CI/CD pipelines, and Git-based source-control workflowsExperience with ETL/ELT, batch and/or streaming processing, production debugging, performance tuning, and data-quality validationFamiliarity with columnar formats such as Parquet and Delta, plus data warehousing and data-analysis conceptsLinux scripting experience (preferred)Exposure to data governance, cataloging, lineage, and discoverability practices (preferred)Experience supporting AI-enabling data assets such as feature tables, vector indices, embeddings, or semantic context (preferred)Experience working in a regulated enterprise environment, preferably healthcare or financial services (preferred)401(k) matching planMedical, dental, and vision plansPaid company holidaysRegular engagement check-ins and consultant supportCompetitive hourly ratesOpportunity to work in a dynamic, innovative environment