{"schemaVersion":"jobsearcher.job.v1","id":"ba8079755262c7389f92d1a2","url":"https://jobsearcher.com/jobs/ba8079755262c7389f92d1a2","canonicalUrl":"https://jobsearcher.com/jobs/ba8079755262c7389f92d1a2","title":"Lead Databricks Developer","description":"Job Title: Lead Databricks Developer\n\nLocation: Wilmington, DE or Plano, TX – 5 Days onsite role\n\nSummary: We are looking for a Lead Databricks Developer with strong experience in Python and Apache Spark to design, develop, and optimize scalable data pipelines and cloud-based data solutions. The ideal candidate will have hands-on expertise in Databricks, PySpark, SQL, and modern data engineering practices.\n\nKey Responsibilities:-\n\nDesign, develop, and maintain ETL/ELT pipelines using Databricks and PySpark- Build scalable batch and real-time data processing solutions using Apache Spark- Develop data transformation workflows using Python, PySpark, and Spark SQL- Implement Delta Lake solutions and optimize data processing performance- Create and manage Databricks notebooks, jobs, and workflows- Ensure data quality, validation, logging, and exception handling- Optimize Spark performance using partitioning, caching, and tuning techniques- Collaborate with cross-functional teams to gather requirements and deliver solutions- Support CI/CD, deployment automation, and production releases- Troubleshoot and resolve issues in data pipelines and platform processes\n\nRequired Skills:-\n\nStrong hands-on experience with Databricks\n\nStrong Python programming skills\n\nStrong in communication and proactive.\n\nExpertise in Apache Spark, PySpark, and Spark SQL- Strong SQL and data transformation skills\n\nExperience with Delta Lake and lakehouse concepts- Knowledge of ETL/ELT and data pipeline design\n\nExperience with AWS, Azure, or GCP- Familiarity with Git, CI/CD, and Agile practices\n\nGood problem-solving and communication skills\n\nPreferred Skills:-\n\nExperience with Airflow, Azure Data Factory, or Databricks Workflows\n\nKnowledge of Kafka or streaming frameworks\n\nExperience with MLflow or analytics platforms\n\nUnderstanding of data governance and security best practices","company":"I8is","rawCompany":"i8is","city":"Wilmington","state":"DE","isRemote":false,"isActive":false,"createdAt":"2026-08-05T00:14:13.811Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"}],"industries":[{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Lead Databricks Developer","description":"Job Title: Lead Databricks Developer\n\nLocation: Wilmington, DE or Plano, TX – 5 Days onsite role\n\nSummary: We are looking for a Lead Databricks Developer with strong experience in Python and Apache Spark to design, develop, and optimize scalable data pipelines and cloud-based data solutions. The ideal candidate will have hands-on expertise in Databricks, PySpark, SQL, and modern data engineering practices.\n\nKey Responsibilities:-\n\nDesign, develop, and maintain ETL/ELT pipelines using Databricks and PySpark- Build scalable batch and real-time data processing solutions using Apache Spark- Develop data transformation workflows using Python, PySpark, and Spark SQL- Implement Delta Lake solutions and optimize data processing performance- Create and manage Databricks notebooks, jobs, and workflows- Ensure data quality, validation, logging, and exception handling- Optimize Spark performance using partitioning, caching, and tuning techniques- Collaborate with cross-functional teams to gather requirements and deliver solutions- Support CI/CD, deployment automation, and production releases- Troubleshoot and resolve issues in data pipelines and platform processes\n\nRequired Skills:-\n\nStrong hands-on experience with Databricks\n\nStrong Python programming skills\n\nStrong in communication and proactive.\n\nExpertise in Apache Spark, PySpark, and Spark SQL- Strong SQL and data transformation skills\n\nExperience with Delta Lake and lakehouse concepts- Knowledge of ETL/ELT and data pipeline design\n\nExperience with AWS, Azure, or GCP- Familiarity with Git, CI/CD, and Agile practices\n\nGood problem-solving and communication skills\n\nPreferred Skills:-\n\nExperience with Airflow, Azure Data Factory, or Databricks Workflows\n\nKnowledge of Kafka or streaming frameworks\n\nExperience with MLflow or analytics platforms\n\nUnderstanding of data governance and security best practices","datePosted":"2026-08-05T00:14:13.811Z","dateModified":"2026-08-05T00:14:13.811Z","hiringOrganization":{"@type":"Organization","name":"I8is","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Wilmington","addressRegion":"DE","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"ba8079755262c7389f92d1a2"},"url":"https://jobsearcher.com/jobs/ba8079755262c7389f92d1a2"}}