Data engineer with Java

Responsibilities: Data Engineer Job Requirements Data Pipeline Expertise - Capable of implementing a diverse range of data solutions with significant autonomy, demonstrating the ability to independently build and maintain complex ETL (Extract, Transform, Load) pipelines Possesses advanced skills in pipeline complexity management and orchestration, with a proven track record of developing high-performance data products accessible via APIs for critical tier 1 trading applications Demonstrates expertise in archiving data products to data lakes for comprehensive analytics purposes Data Quality and Advanced Analytics - Robust proficiency in Data Quality, Data Validation, and Anomaly Detection techniques - Experience with AI/ML and Large Language Model (LLM) approaches to data analysis and validation - Ability to implement sophisticated data integrity and verification processes Qualifications Technical Skill Requirements: Programming and Cloud Technologies - Expert-level skills in: - Python - SQL - AWS Cloud Services: - Cloud Storage - Lambda - ECS - Glue Data Quality - HDMI - Redshift Machine Learning - OpenSearch - RDS API and Resilience Technologies - Advanced skills in building resilient API architectures, including: - Microservices design patterns - API gateway configuration - Circuit breaker implementations - Fault tolerance and error handling mechanisms - Expertise in API development and management tools: - Swagger/OpenAPI - Postman - GraphQL - RESTful API design principles Proficiency in resilience engineering techniques: - Distributed system design - Load balancing - Failover strategies - Horizontal and vertical scaling approaches - Containerization and orchestration: - Docker - Kubernetes - Helm charts Monitoring and observability: - Prometheus - Grafana - ELK Stack (Elasticsearch, Logstash, Kibana) - Distributed tracing with Jaeger or Zipkin Additional Technical Competencies Proficient in Apache Flink - Advanced Java programming skills - Proven ability to create and optimize complex data products - Demonstrated Subject Matter Expertise (SME) in data engineering principles ## Professional Attributes Capability to work independently through complex data challenges - Robust preference for candidates with prior experience in investment data domain and related technologies Key Differentiators - Deep understanding of data engineering best practices - Ability to design scalable, efficient, and resilient data solutions - Proven track record of solving intricate data infrastructure problems with robust API architectures Benefits at IntelliBee Long-Term Stability: Join us on a multi-year opportunities with room to grow. Comprehensive Health Coverage: Access quality healthcare benefits to keep you and your family well. Future Planning: Enroll in our 401(k) program and invest in your financial security. GC Assistance: We support immediate Green Card processing, if required.