{"schemaVersion":"jobsearcher.job.v1","id":"07f385aedc1904c077de80b0","url":"https://jobsearcher.com/jobs/07f385aedc1904c077de80b0","canonicalUrl":"https://jobsearcher.com/jobs/07f385aedc1904c077de80b0","title":"Lead Data Engineer","description":"Lead Data EngineerWe are seeking an experienced Lead Data Engineer with strong expertise in AWS, Databricks, and modern cloud data engineering to lead the design, development, and optimization of scalable data platforms. The ideal candidate will have 10+ years of experience building enterprise data solutions, leading technical teams, and implementing high-performance data pipelines using Databricks on AWS.Key ResponsibilitiesLead the design, development, and maintenance of enterprise-scale data platforms on AWS and and implement scalable ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Workflows.Build and optimize Delta Lake architectures, ensuring data quality, reliability, and performance.Design and implement data ingestion pipelines for structured, semi-structured, and streaming with business stakeholders, architects, analysts, and data scientists to gather requirements and deliver data solutions.Optimize Spark jobs, SQL queries, and cluster configurations for performance and cost efficiency.Implement data governance, security, and access controls using Unity reusable frameworks, coding standards, and best practices for data engineering.Lead code reviews, mentor junior engineers, and provide technical leadership.Build CI/CD pipelines and automate deployment processes.Monitor production jobs, troubleshoot failures, and ensure SLA compliance.Support cloud migration and modernization initiatives from legacy systems to AWS Skills10+ years of experience in Data Engineering.4+ years of hands-on experience with experience with AWS Cloud services including: S3, EMR, Glue, Lambda, IAM, CloudWatch, Redshift, RDS, EC2.Expertise in PySpark, Spark SQL, and Python.Strong SQL development and query optimization skills.Experience with Delta Lake, Delta Live Tables (DLT), and Unity building batch and streaming data pipelines.Knowledge of Apache Spark architecture and performance tuning.Experience with Git, CI/CD, and DevOps practices.Familiarity with Agile/Scrum methodologies.","company":"eSolutions","rawCompany":"esolutions","state":"FL","isRemote":false,"isActive":false,"createdAt":"2026-09-11T03:55:55.574Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Lead Data Engineer","description":"Lead Data EngineerWe are seeking an experienced Lead Data Engineer with strong expertise in AWS, Databricks, and modern cloud data engineering to lead the design, development, and optimization of scalable data platforms. The ideal candidate will have 10+ years of experience building enterprise data solutions, leading technical teams, and implementing high-performance data pipelines using Databricks on AWS.Key ResponsibilitiesLead the design, development, and maintenance of enterprise-scale data platforms on AWS and and implement scalable ETL/ELT pipelines using PySpark, Spark SQL, and Databricks Workflows.Build and optimize Delta Lake architectures, ensuring data quality, reliability, and performance.Design and implement data ingestion pipelines for structured, semi-structured, and streaming with business stakeholders, architects, analysts, and data scientists to gather requirements and deliver data solutions.Optimize Spark jobs, SQL queries, and cluster configurations for performance and cost efficiency.Implement data governance, security, and access controls using Unity reusable frameworks, coding standards, and best practices for data engineering.Lead code reviews, mentor junior engineers, and provide technical leadership.Build CI/CD pipelines and automate deployment processes.Monitor production jobs, troubleshoot failures, and ensure SLA compliance.Support cloud migration and modernization initiatives from legacy systems to AWS Skills10+ years of experience in Data Engineering.4+ years of hands-on experience with experience with AWS Cloud services including: S3, EMR, Glue, Lambda, IAM, CloudWatch, Redshift, RDS, EC2.Expertise in PySpark, Spark SQL, and Python.Strong SQL development and query optimization skills.Experience with Delta Lake, Delta Live Tables (DLT), and Unity building batch and streaming data pipelines.Knowledge of Apache Spark architecture and performance tuning.Experience with Git, CI/CD, and DevOps practices.Familiarity with Agile/Scrum methodologies.","datePosted":"2026-09-11T03:55:55.574Z","dateModified":"2026-09-11T03:55:55.574Z","hiringOrganization":{"@type":"Organization","name":"eSolutions","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressRegion":"FL","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"07f385aedc1904c077de80b0"},"url":"https://jobsearcher.com/jobs/07f385aedc1904c077de80b0"}}