Data Engineer

We're hiring on behalf of a Haystack partner!The Role• Design, build, and maintain scalable batch and streaming data pipelines using Apache Spark and modern cloud data platforms.• Develop and optimize ETL/ELT workflows to ingest, transform, and curate data from diverse source systems.• Implement data modeling and transformation logic to support reporting, dashboards, and downstream analytical workloads.• Build and manage data processing workloads within modern lakehouse platforms, with a preference for Microsoft Fabric/OneLake.• Ensure data quality, reliability, and consistency through validation, monitoring, and reconciliation processes.• Optimize Spark jobs for performance, cost efficiency, and scalability.What You'll Need• Strong experience as a Data Engineer building and operating production data pipelines.• Hands-on experience with Apache Spark for large-scale data processing.• Proficiency in Python, SQL, and data transformation best practices.• Experience with cloud-based data platforms and storage, including Data Lakes and Lakehouse architectures.• Familiarity with Microsoft Fabric, OneLake, or similar analytics platforms is a strong plus.• Experience designing and optimizing data models for analytical workloads.What's On Offer• Competitive annual salary of $90,700 - $153,925 plus potential for connectivity reimbursement.• Comprehensive technology setup including laptop, monitors, headset, keyboard, and mouse.• Opportunity to work with modern cloud data platforms and cutting-edge technologies.• A role critical to enabling data-driven decision-making within a large organization.Apply via Haystack today!