Lead Cloud Data Engineer
Job Title: Lead Cloud Data EngineerLocations: Fort Mill, SC or New York City, NY (Hybrid/Onsite)Type: Full TimeJob SummaryWe are looking for a highly experienced Data Engineer - AWS Tech Lead to design and lead scalable cloud-based data solutions. This role will combine hands-on engineering with leadership responsibilities, guiding a team of data engineers while collaborating with architects, product owners, and business stakeholders. The ideal candidate will have deep expertise in AWS data services, ETL/ELT pipelines, and big data frameworks, with proven experience in leading technical delivery. Key ResponsibilitiesLead the design, development, and deployment of large-scale data pipelines and solutions on AWSProvide technical leadership and mentorship to a team of data engineers, ensuring best practices in coding, architecture, and performance optimizationArchitect data lake and data warehouse solutions leveraging AWS Glue, Redshift, S3, Athena, EMR, and LambdaDrive development of ETL/ELT processes using PySpark, Python, and SQLImplement and enforce data governance, quality checks, and security standards across the platformCollaborate with business analysts, data scientists, and product teams to translate business requirements into scalable solutionsOptimize cost and performance of AWS-based data workloadsTroubleshoot production issues and ensure high availability of data pipelinesParticipate in sprint planning, backlog refinement, and Agile ceremonies as a technical leadRequired Qualifications13+ years of experience in Data Engineering, with at least 4+ years in a technical leadership roleStrong expertise in AWS data services: Glue, S3, Redshift, Athena, Lambda, Step Functions, EMRProficiency in PySpark, Python, and SQL for large-scale data processingSolid experience in data modeling (star, snowflake, dimensional modeling) and data lake/warehouse architecturesProven track record in leading and mentoring data engineering teamsStrong understanding of Agile methodologies and CI/CD practices for data pipelinesPreferred QualificationsExperience in BFSI / Wealth Management domainAWS Certification (Data Analytics Specialty, Solutions Architect, or Big Data)Familiarity with containerization (Docker, Kubernetes) and orchestration frameworksEducationBachelor's degree in Computer Science, Information Technology, Engineering, or related fieldMaster's degree preferred