JOBSEARCHER

Senior Data Engineer

ARCHIVED

We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.

An established enterprise organization is seeking a Senior Data Engineer for a long-term contract opportunity supporting its cloud-based data platform. This is a hands-on individual contributor role focused primarily on maintaining, troubleshooting, and optimizing existing enterprise data pipelines, with additional responsibility for new development and enhancements. The ideal candidate will bring strong production experience with PySpark/Spark, Databricks, Python, and enterprise data pipelines and be comfortable working independently within an established Azure lakehouse environment. The position is fully remote within the U.S. and requires availability during Eastern Time business hours.
Responsibilities: Maintain, troubleshoot, optimize, and enhance existing enterprise data pipelines. Develop scalable data pipelines and transformations using Python, PySpark/Spark, and Databricks. Ingest and transform data across raw, transformed, and curated lakehouse layers. Monitor pipeline performance, reliability, dependencies, and resource consumption. Identify inefficient workloads and implement improvements to processing performance and cloud costs. Develop smaller new pipelines, enhancements, and data integrations as needed. Collaborate with data architects, engineers, reporting teams, and offshore development/support resources. Produce clear technical documentation covering code, pipelines, dependencies, processes, and technical decisions. Support an established production data environment while following existing architecture and development standards.
Requirements: 7+ years of Data Engineering experience, ideally at the Senior Data Engineer level. Strong, hands-on production experience with PySpark or Apache Spark. Strong hands-on development experience with Databricks. Advanced Python development skills. Demonstrated experience building, maintaining, troubleshooting, and optimizing enterprise data pipelines. Strong understanding of distributed data processing and lakehouse/data-lake architectures. Experience working within an established production codebase and data platform. Strong technical documentation and communication skills. Ability to work independently while collaborating effectively across technical teams. Azure and Azure Data Factory experience preferred. Familiarity with Microsoft Fabric is a plus. Must be based in the U.S. and available to work Eastern Time business hours.