Lead Data Engineer – AWS & Big Data

Job SummaryWe are seeking a highly experienced Lead Data Engineer to design, develop, and manage scalable data platforms and pipelines on AWS. The ideal candidate will have strong expertise in big data technologies, cloud architecture, data modeling, and ETL/ELT development, along with exposure to AI/ML integration and BI tools.You will play a key role in building robust data infrastructure that supports analytics, reporting, and advanced data-driven decision-making across the organization.Key ResponsibilitiesDesign, build, and maintain scalable batch and real-time data pipelines using AWS servicesDevelop and optimize data lakes and data warehouses using Amazon S3, Redshift, and GlueBuild efficient ETL/ELT pipelines using Python, PySpark, and SQLImplement data modeling techniques such as Star Schema, Snowflake Schema, and Dimensional ModelingWork with distributed computing frameworks like Hadoop and Apache SparkAutomate infrastructure provisioning using Terraform (Infrastructure as Code)Ensure data quality, governance, security, and compliance across pipelinesIntegrate AI/ML models into data pipelines to enable advanced analytics use casesCollaborate with data scientists, analysts, and business stakeholders to deliver data solutionsDevelop dashboards and reports using Power BIMonitor, troubleshoot, and optimize performance of data pipelines and cloud resourcesRequired Skills & Experience12–15 years of experience in Data Engineering / Big Data / Cloud Data PlatformsStrong hands-on experience with AWS services:Amazon S3Amazon RedshiftAWS GlueAWS LambdaEMRAthenaStrong proficiency in Python, PySpark, and SQLExperience with Hadoop and Apache Spark ecosystemsStrong knowledge of data warehousing and dimensional data modelingHands-on experience with Terraform (IaC)Understanding of AI/ML concepts and integration into data pipelinesExperience with Power BI or similar BI tools