{"schemaVersion":"jobsearcher.job.v1","id":"e079395af51a84e6a011a253","url":"https://jobsearcher.com/jobs/e079395af51a84e6a011a253","canonicalUrl":"https://jobsearcher.com/jobs/e079395af51a84e6a011a253","title":"Databricks Data Engineering Lead","description":"HomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsHomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsJob DescriptionJob OverviewJob ID:J54660Job Title:Databricks Data Engineering LeadLocation:Albany, NYDuration:8 Months + ExtensionDomain Exposure:TelecomClient:To Be Discussed LaterEmployment Type:W-2, 1099, C2CWork Authorization:US Citizen, Green Card, H-1B, OPT-EAD, CPT, OtherView All JobsSimilar JobsDatabricks Engineer - Austin, TXSenior Databricks Platform Administrator - Weehawken, NJDatabricks Architect - Princeton, NJDatabricks Engineer - Austin, TXDatabricks Admin - Branchville, NJJob Title: Databricks Data Engineering LeadLocation: Albany,New YorkDuration: 8 Months + ExtensionBill Rate: $87/hour (C2C)Job Type: W-2, 1099, C2CClient: To Be Discussed LaterWork Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, OtherJob Description:We are looking for an experienced Databricks Data Engineering Lead with 8+ years of overall data engineering experience and strong hands-on expertise in building scalable data solutions using Databricks, Python, PySpark, Delta Lake, and Databricks Genie.The candidate will be responsible for leading the design and development of modern data engineering solutions, providing technical direction to the engineering team, and ensuring scalable, secure, high-performing implementations. Experience working with Databricks on AWS is highly desirable.Key ResponsibilitiesLead the design, development, and implementation of enterprise-scale data engineering solutions on Databricks.Design and develop scalable ETL/ELT pipelines using Databricks, Python, and PySpark.Provide hands-on technical leadership for Databricks Genie implementations and use cases.Design and optimize data architectures leveraging Delta Lake and Lakehouse architectural patterns.Build reusable frameworks and engineering components for data ingestion, transformation, validation, and processing.Develop and optimize complex PySpark workloads for performance, reliability, and scalability.Implement data quality, governance, security, monitoring, and operational controls across data pipelines.Provide technical leadership through architecture reviews, design reviews, code reviews, troubleshooting, and engineering best practices.Mentor data engineers and provide technical guidance during development and implementation.Work closely with architects, business analysts, data teams, and application teams to translate requirements into technical solutions.Support production deployments, performance tuning, incident resolution, and continuous improvement.Contribute to technical documentation, solution design, deployment strategy, and engineering standards.Mandatory Technical Skills8+ years of overall experience in Data Engineering / Data Platforms.Strong hands-on experience with Databricks.Strong practical experience with Databricks Genie.Advanced programming experience in Python.Strong development and optimization experience using PySpark.Strong understanding and implementation experience with Delta Lake.Experience designing and implementing Lakehouse-based data platforms.Strong knowledge of SQL and data modelling concepts.Experience developing enterprise-grade batch and/or streaming data pipelines.Strong understanding of data quality, observability, performance tuning, and production support.Experience leading technical teams and driving engineering standards and best practices.Desired / Preferred SkillsHands-on experience implementing Databricks workloads on AWS.Experience with AWS data services such as Amazon S3, AWS IAM, AWS Glue, Amazon Redshift, AWS Lambda, and Amazon CloudWatch.Knowledge of CI/CD and DevOps practices for data engineering solutions.Exposure to Databricks governance and security capabilities.Experience working on large-scale data modernization or cloud migration programs.Experience working directly with business and customer stakeholders.Apply NowEqual Opportunity EmployerWe are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.Home|About|Adaptive AI|Industries|Consulting |Contact Us© 2004-2026, Machine Learning Technologies LLC, or its affiliates","company":"Machine Learning Technologies","rawCompany":"machine learning technologies","city":"Albany","state":"NY","isRemote":false,"isActive":false,"createdAt":"2026-11-12T09:41:19.066Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Databricks Data Engineering Lead","description":"HomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsHomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsJob DescriptionJob OverviewJob ID:J54660Job Title:Databricks Data Engineering LeadLocation:Albany, NYDuration:8 Months + ExtensionDomain Exposure:TelecomClient:To Be Discussed LaterEmployment Type:W-2, 1099, C2CWork Authorization:US Citizen, Green Card, H-1B, OPT-EAD, CPT, OtherView All JobsSimilar JobsDatabricks Engineer - Austin, TXSenior Databricks Platform Administrator - Weehawken, NJDatabricks Architect - Princeton, NJDatabricks Engineer - Austin, TXDatabricks Admin - Branchville, NJJob Title: Databricks Data Engineering LeadLocation: Albany,New YorkDuration: 8 Months + ExtensionBill Rate: $87/hour (C2C)Job Type: W-2, 1099, C2CClient: To Be Discussed LaterWork Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, OtherJob Description:We are looking for an experienced Databricks Data Engineering Lead with 8+ years of overall data engineering experience and strong hands-on expertise in building scalable data solutions using Databricks, Python, PySpark, Delta Lake, and Databricks Genie.The candidate will be responsible for leading the design and development of modern data engineering solutions, providing technical direction to the engineering team, and ensuring scalable, secure, high-performing implementations. Experience working with Databricks on AWS is highly desirable.Key ResponsibilitiesLead the design, development, and implementation of enterprise-scale data engineering solutions on Databricks.Design and develop scalable ETL/ELT pipelines using Databricks, Python, and PySpark.Provide hands-on technical leadership for Databricks Genie implementations and use cases.Design and optimize data architectures leveraging Delta Lake and Lakehouse architectural patterns.Build reusable frameworks and engineering components for data ingestion, transformation, validation, and processing.Develop and optimize complex PySpark workloads for performance, reliability, and scalability.Implement data quality, governance, security, monitoring, and operational controls across data pipelines.Provide technical leadership through architecture reviews, design reviews, code reviews, troubleshooting, and engineering best practices.Mentor data engineers and provide technical guidance during development and implementation.Work closely with architects, business analysts, data teams, and application teams to translate requirements into technical solutions.Support production deployments, performance tuning, incident resolution, and continuous improvement.Contribute to technical documentation, solution design, deployment strategy, and engineering standards.Mandatory Technical Skills8+ years of overall experience in Data Engineering / Data Platforms.Strong hands-on experience with Databricks.Strong practical experience with Databricks Genie.Advanced programming experience in Python.Strong development and optimization experience using PySpark.Strong understanding and implementation experience with Delta Lake.Experience designing and implementing Lakehouse-based data platforms.Strong knowledge of SQL and data modelling concepts.Experience developing enterprise-grade batch and/or streaming data pipelines.Strong understanding of data quality, observability, performance tuning, and production support.Experience leading technical teams and driving engineering standards and best practices.Desired / Preferred SkillsHands-on experience implementing Databricks workloads on AWS.Experience with AWS data services such as Amazon S3, AWS IAM, AWS Glue, Amazon Redshift, AWS Lambda, and Amazon CloudWatch.Knowledge of CI/CD and DevOps practices for data engineering solutions.Exposure to Databricks governance and security capabilities.Experience working on large-scale data modernization or cloud migration programs.Experience working directly with business and customer stakeholders.Apply NowEqual Opportunity EmployerWe are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.Home|About|Adaptive AI|Industries|Consulting |Contact Us© 2004-2026, Machine Learning Technologies LLC, or its affiliates","datePosted":"2026-11-12T09:41:19.066Z","dateModified":"2026-11-12T09:41:19.066Z","hiringOrganization":{"@type":"Organization","name":"Machine Learning Technologies","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Albany","addressRegion":"NY","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"e079395af51a84e6a011a253"},"url":"https://jobsearcher.com/jobs/e079395af51a84e6a011a253"}}