Senior Data Engineer

We are seeking a Senior Data Engineer with strong expertise in AWS-based data engineering, real-time streaming technologies, and enterprise-grade data quality frameworks. The ideal candidate will design, build, and optimize scalable batch and streaming data pipelines, implement robust data validation and monitoring processes, and support mission-critical analytics platforms.Key Responsibilities:Develop and maintain scalable ETL/ELT pipelines using AWS Glue, PySpark, and PythonDesign and manage real-time streaming solutions using Kafka, KSQL, and Apache FlinkImplement and enforce comprehensive data quality frameworks, including validation, profiling, monitoring, and reconciliationOptimize data processing performance, scalability, reliability, and cost in cloud environmentsCollaborate with cross-functional teams to deliver reliable, production-grade data platforms and ensure data integrity across the pipelineRequired Skills:Strong hands-on experience with Python and PySparkProven expertise in AWS Glue, Lambda, and other cloud-native data servicesSolid experience with the Kafka ecosystem (topics, partitions, consumer groups, streaming patterns)Demonstrated experience building and supporting data quality frameworks (validation rules, reconciliation checks, profiling, anomaly detection)Strong understanding of distributed data processing and scalable architecture patternsGood-to-Have Skills:Experience with Apache Flink for real-time stream processing and stateful computationsKnowledge of KSQL or other streaming SQL enginesExposure to CI/CD pipelines, IaC (Terraform/CloudFormation), and DevOps practicesFamiliarity with data lake/lakehouse architectures and table formats such as Iceberg, Delta, or HudiExperience working in enterprise or financial data environmentsJ-18808-Ljbffr