{"schemaVersion":"jobsearcher.job.v1","id":"f274975f6233b3deaf61254f","url":"https://jobsearcher.com/jobs/f274975f6233b3deaf61254f","canonicalUrl":"https://jobsearcher.com/jobs/f274975f6233b3deaf61254f","title":"Data Engineer","description":"Azure Databricks Technical LeadWe are looking for a highly skilled Senior Azure Databricks (ADB) Developer to join our Data Engineering team. This role involves developing large-scale batch and streaming data pipelines on Azure Cloud. The ideal candidate will have strong expertise in Python, Databricks Notebooks, Apache Spark (including Structured Streaming), and real-time integration with Kafka. You will work with both relational databases like DB2 and NoSQL systems such as MongoDB, focusing on performance optimization and scalable architecture.Key Responsibilities:Design and Develop: Create real-time and batch data pipelines using Azure Databricks, Apache Spark, and Structured Streaming.Data Processing: Write efficient ETL scripts and automate workflows using Python.Data Integration: Integrate with various data sources and destinations, including DB2, MongoDB, and other enterprise-grade data systems.Performance Optimization: Tune Spark jobs for optimal performance and cost-effective compute usage on Azure.Collaboration: Work with platform and architecture teams to ensure secure, scalable, and maintainable cloud data infrastructure.CI/CD Support: Implement CI/CD for Databricks pipelines and notebooks using tools like GitHub and Azure DevOps.Stakeholder Communication: Interface with product owners, data scientists, and business analysts to translate data requirements into production-ready pipelines.","company":"Intersources","rawCompany":"intersources","city":"Pleasanton","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-08-25T04:09:53.864Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1243.00","title":"Database Architects","slug":"database-architects"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer","description":"Azure Databricks Technical LeadWe are looking for a highly skilled Senior Azure Databricks (ADB) Developer to join our Data Engineering team. This role involves developing large-scale batch and streaming data pipelines on Azure Cloud. The ideal candidate will have strong expertise in Python, Databricks Notebooks, Apache Spark (including Structured Streaming), and real-time integration with Kafka. You will work with both relational databases like DB2 and NoSQL systems such as MongoDB, focusing on performance optimization and scalable architecture.Key Responsibilities:Design and Develop: Create real-time and batch data pipelines using Azure Databricks, Apache Spark, and Structured Streaming.Data Processing: Write efficient ETL scripts and automate workflows using Python.Data Integration: Integrate with various data sources and destinations, including DB2, MongoDB, and other enterprise-grade data systems.Performance Optimization: Tune Spark jobs for optimal performance and cost-effective compute usage on Azure.Collaboration: Work with platform and architecture teams to ensure secure, scalable, and maintainable cloud data infrastructure.CI/CD Support: Implement CI/CD for Databricks pipelines and notebooks using tools like GitHub and Azure DevOps.Stakeholder Communication: Interface with product owners, data scientists, and business analysts to translate data requirements into production-ready pipelines.","datePosted":"2026-08-25T04:09:53.864Z","dateModified":"2026-08-25T04:09:53.864Z","hiringOrganization":{"@type":"Organization","name":"Intersources","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Pleasanton","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"f274975f6233b3deaf61254f"},"url":"https://jobsearcher.com/jobs/f274975f6233b3deaf61254f"}}