Lead Senior Java Developer

Location: Jersey City, NJ (Hybrid Onsite-4 Days Onsite Per Week)

Duration: Fulltime

Job Description: We are seeking a highly skilled and experienced Senior Java Engineer to join our team. The ideal candidate will have strong expertise in Java, Spring boot, Microservices, Kafka, AWS, Kubernetes, Docker, JUnit, Maven/GradlePython, PySpark, Apache Iceberg, Flink, and Kafka, with additional exposure to API development and integration. This role involves designing, building, and optimizing scalable data pipelines and systems to support our data-driven initiatives.

Key Responsibilities: Data Pipeline Development:

• Design, develop, and maintain scalable and efficient data pipelines using
• PySpark Java, Spring boot, Microservices, Kafka, AWS, Kubernetes, Docker, JUnit, Maven/Gradle.
• Implement real-time and batch data processing workflows leveraging Apache Flink and Kafka.
• Data Lakehouse Management:
• Work with Apache Iceberg to manage large-scale datasets, including schema evolution, partitioning, and time-travel queries.
• Optimize data storage and retrieval for performance and scalability.
• API Development and Integration:
• Develop and integrate APIs to enable seamless data access and interaction with downstream systems.
• Ensure APIs are secure, scalable, and optimized for performance.
• Data Streaming and Messaging:
• Build and manage real-time data streaming solutions using Kafka.
• Implement event-driven architectures to support business use cases.
• Data Governance and Quality:
• Ensure data quality, consistency, and security across all pipelines and systems.
• Implement monitoring and alerting mechanisms to track data pipeline performance and reliability.
• Collaboration and Mentorship:
• Collaborate with cross-functional teams, including data scientists, analysts, and DevOps engineers.
• Mentor junior engineers and contribute to best practices in data engineering.

Required Skills: Programming and Frameworks:

• Proficiency in JavaStrong understanding of distributed data processing frameworks.
• Big Data Technologies:
• Hands-on experience with Apache Iceberg for managing large-scale datasets.
• Expertise in Apache Flink for real-time data processing.
• Proficiency in Kafka for data streaming and messaging.
• API Development:
• Experience in developing and integrating RESTful APIs.
• Knowledge of API security, scalability, and performance optimization.
• Data Engineering:
• Strong understanding of ETL/ELT processes and data transformation techniques.
• Familiarity with data lakehouse architectures and best practices.
• DevOps and CI/CD:Experience with CI/CD pipelines for deploying and managing data pipelines.
• Familiarity with containerization tools like Docker and orchestration platforms like Kubernetes.

Preferred Skills: Java, Spring boot, Microservices, Kafka, AWS, Kubernetes, Docker, JUnit, Maven/Gradle Knowledge of data governance frameworks and compliance requirements. Familiarity with monitoring tools like Prometheus, Grafana, or similar. Experience with cloud platforms (AWS, Azure, or GCP) for application engineering solutions.