Java Spark Engineer

**Will prioritize candidates in New York but will consider remote candidates**About the Job:As a member of the development team, you will help define, establish, and evolve our development capacities. You will be tasked in creating innovative solutions that advance our clients businesses. You'll join a strong and inspiring team of technologists dedicated to improving the design, analytics, development, coding, testing and application programming that goes into creating high-quality software and new products. You'll be tasked with keeping the team and other key stakeholders up to speed on the progress of what's being developed.What You'll Get to Do:You will contribute to creating one of our most ambitious and critical technology projects. You will participate in technical and architectural conversations with peers and experts to design the platform together. Additionally, will work with a team of other highly motivated data engineers, product managers, SDETs, etc.Work collaboratively across the organizationSolve challenging problems in an elegant but pragmatic wayWrite high quality, reusable code and adhere to coding standardsWrite professional, clear, and comprehensive documentationApply best development and DevOps practices Be aware of on-going technology trends and know how to leverage their benefitsWhat you'll bring with You:5+ years experience using Java and/or Scala4+ years of Apache Spark is a mustMicrosoft Azure is a must / ADF experience is a plus Solid understanding of Spark/Databricks batch and streaming solutions (Delta Lake, Lakehouse) is a plusImplemented robust products, services and capabilities for the data platformKnowledge of Azure Data FactoryKnowledge of Kafka, Event hubs and ADLS Gen2 on Microsoft AzureKnowledge of TDD, BDD and how to integrate within the technologies described above