Lead Data Engineer

Job Summary:We are seeking a highly experienced Lead Data Engineer to help drive the next evolution of our enterprise data, analytics, and AI ecosystem. In this role, you will lead the design, development, and optimization of a modern AWS + Databricks Lakehouse platform while partnering closely with business and technical stakeholders to deliver scalable, high-performance data solutions.This is a hands-on technical leadership role requiring expertise in data engineering, cloud technologies, ETL/ELT, and modern data architecture.Key ResponsibilitiesDesign, develop, and maintain enterprise-grade batch and streaming data pipelines.Lead the full ETL/ELT lifecycle from design through deployment and production support.Build scalable, secure, and high-performance data solutions using AWS and Databricks.Drive data engineering best practices, architecture standards, and governance.Optimize Databricks workloads for performance and cost efficiency.Implement and manage Delta Lake, Unity Catalog, Delta Live Tables (DLT), Databricks SQL, and workflow orchestration.Collaborate with business stakeholders to translate requirements into scalable technical solutions.Mentor junior engineers while remaining hands-on with development.Ensure data quality, reliability, governance, and operational excellence.Participate in Agile ceremonies and contribute to continuous improvement initiatives.Required Qualifications10+ years of IT experience with strong expertise in Data Engineering and ETL.4–6+ years of recent hands-on experience designing and building large-scale data pipelines.Strong experience with:DatabricksApache SparkDelta LakeUnity CatalogDelta Live Tables (DLT)Databricks SQLStrong AWS experience including:Amazon S3IAMNetworkingEC2Expertise in SQL, Python, data modeling, and distributed data processing.Experience with Lakehouse architecture and Medallion architecture.Experience with CI/CD, DevOps, and workflow orchestration.Strong understanding of Databricks performance tuning and cost optimization, including:Cluster sizingAuto ScalingSpot InstancesPhoton Engine optimizationData shuffle optimizationExperience designing, configuring, coding, testing, and deploying enterprise data platforms.Strong communication skills with the ability to work directly with business stakeholders.Experience working in Agile environments.Preferred SkillsLeadership or mentoring experience.Strong analytical and problem-solving skills.Experience supporting enterprise analytics and AI initiatives.Passion for learning new technologies and modern data platforms.Technical SkillsDatabricksApache SparkDelta LakeUnity CatalogDelta Live Tables (DLT)Databricks SQLAWS (S3, IAM, EC2, Networking)PythonSQLETL / ELTData ModelingDistributed ProcessingLakehouse ArchitectureMedallion ArchitectureCI/CDDevOpsAgileInterview Process60–90 minute technical interview focused on architecture, Databricks, AWS, and hands-on design.30–45 minute behavioral/team fit interview with the hiring manager.