Java Spark Engineer

JOB DESCRIPTION Role: Java Spark Engineer Location: Berkley Heights, NJ Work Mode: 5-days in office (flexible to support weekend) Responsibilities : • Architect and build scalable, fault-tolerant data pipelines using Apache Spark (Java) • Lead design of batch and streaming ETL/ELT systems handling large data volumes • Deep-dive performance tuning: partitioning strategy, memory management, shuffle/skew optimization, job cost reduction • Set coding standards and lead code/design reviews across the team • Drive technical decisions on data architecture, storage formats, and pipeline orchestration • Mentor mid-level and junior engineers; act as a technical escalation point • Partner with product, analytics, and platform teams to translate requirements into scalable systems • Own production reliability — on-call ownership, incident response, root-cause analysis for pipeline failures • Evaluate and introduce new tools/frameworks where they improve the system • Contribute to capacity planning and cost optimization for cluster infrastructure Required Qualifications • Bachelor’s or Master’s degree in Computer Science, Engineering, or related field • 7+ years of professional Java development experience • 5+ years hands-on experience with Apache Spark in production environments • Expert-level understanding of distributed systems: fault tolerance, data locality, shuffle mechanics, resource management • Proven track record designing systems processing terabyte+ scale data • Strong SQL skills and deep familiarity with columnar storage formats (Parquet, ORC, Avro, Delta Lake/Iceberg) • Experience with cluster managers (YARN, Kubernetes) and cloud-managed Spark • Proficiency with Kafka • Strong grasp of CI/CD, containerization, and infrastructure-as-code practices Preferred Qualifications • Experience with Flink or other stream-processing frameworks • Familiarity with data governance, lineage, and quality frameworks • Experience with workflow orchestration at scale  • Background in system design for multi-tenant or multi-region data platforms • Prior experience leading a team or acting as a technical lead Soft Skills / Leadership • Excellent communication — able to explain technical tradeoffs to non-technical stakeholders • Strong mentorship and coaching ability • Comfortable driving ambiguous, cross-team technical initiatives Behavioral Skills: Good Communication skills 5 days Work from Office at Berkley Heights, NJ Team Player Ability to work in a changing environment Strong problem solving and analytical skills Ability to work independently or within a team