{"schemaVersion":"jobsearcher.job.v1","id":"daacdb2f786503da84da035d","url":"https://jobsearcher.com/jobs/daacdb2f786503da84da035d","canonicalUrl":"https://jobsearcher.com/jobs/daacdb2f786503da84da035d","title":"Data Engineer - Databricks / Spark / AI","description":"Senior Data Engineer Databricks / Spark / AI\n\nLocation: San Francisco, CA\nWork Model: Hybrid 2 Days/Week Onsite\nDuration: 12+ Months\nEmployment Type: Contract\nExperience: 5+ Years\nWork Authorization: Visa-independent candidates required\n\nJob Summary\n\nWe are seeking a highly skilled Senior Data Engineer with strong, recent hands-on experience in Databricks, Apache Spark, SQL, and Python.\n\nThis is primarily a Data Engineering role with additional exposure to AI and Generative AI capabilities within the Databricks ecosystem. The ideal candidate will have extensive experience designing and developing scalable data pipelines and data processing solutions using Databricks and Spark.\n\nCandidates should also have hands-on experience with Databricks Genie and an understanding of how AI capabilities can be applied to enterprise data and analytics solutions.\n\nStrong, recent Databricks experience is essential for this position.\n\nKey Responsibilities\n\nDesign, develop, and maintain scalable data engineering solutions using Databricks, Apache Spark, SQL, and Python.\n\nBuild and optimize high-volume ETL/ELT data pipelines and data-processing workflows.\n\nDevelop complex SQL queries and transformations for large-scale datasets.\n\nBuild distributed data-processing solutions using PySpark/Spark.\n\nDesign reliable and scalable data architectures within the Databricks ecosystem.\n\nWork hands-on with Databricks Genie to enable AI-powered conversational data and analytics capabilities.\n\nIntegrate AI/GenAI capabilities with enterprise data platforms and analytics workflows.\n\nOptimize Databricks workloads for performance, scalability, reliability, and cost efficiency.\n\nPerform data transformation, cleansing, validation, and quality checks.\n\nTroubleshoot performance issues across Spark jobs, SQL workloads, pipelines, and Databricks environments.\n\nWork with structured and semi-structured datasets from multiple enterprise sources.\n\nCollaborate with Data Engineering, Analytics, AI/ML, Product, and business teams.\n\nTranslate business and analytical requirements into scalable data solutions.\n\nParticipate in technical design, code reviews, testing, deployment, and production support.\n\nMaintain engineering standards, documentation, and data-development best practices.\n\nParticipate in Agile/Scrum development activities using Jira and Confluence.\n\nMust-Have Skills\n\nCandidates should demonstr","company":"Innova Software Services","rawCompany":"innova software services","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-09-26T09:00:11.806Z","occupations":[{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Data Engineer - Databricks / Spark / AI","description":"Senior Data Engineer Databricks / Spark / AI\n\nLocation: San Francisco, CA\nWork Model: Hybrid 2 Days/Week Onsite\nDuration: 12+ Months\nEmployment Type: Contract\nExperience: 5+ Years\nWork Authorization: Visa-independent candidates required\n\nJob Summary\n\nWe are seeking a highly skilled Senior Data Engineer with strong, recent hands-on experience in Databricks, Apache Spark, SQL, and Python.\n\nThis is primarily a Data Engineering role with additional exposure to AI and Generative AI capabilities within the Databricks ecosystem. The ideal candidate will have extensive experience designing and developing scalable data pipelines and data processing solutions using Databricks and Spark.\n\nCandidates should also have hands-on experience with Databricks Genie and an understanding of how AI capabilities can be applied to enterprise data and analytics solutions.\n\nStrong, recent Databricks experience is essential for this position.\n\nKey Responsibilities\n\nDesign, develop, and maintain scalable data engineering solutions using Databricks, Apache Spark, SQL, and Python.\n\nBuild and optimize high-volume ETL/ELT data pipelines and data-processing workflows.\n\nDevelop complex SQL queries and transformations for large-scale datasets.\n\nBuild distributed data-processing solutions using PySpark/Spark.\n\nDesign reliable and scalable data architectures within the Databricks ecosystem.\n\nWork hands-on with Databricks Genie to enable AI-powered conversational data and analytics capabilities.\n\nIntegrate AI/GenAI capabilities with enterprise data platforms and analytics workflows.\n\nOptimize Databricks workloads for performance, scalability, reliability, and cost efficiency.\n\nPerform data transformation, cleansing, validation, and quality checks.\n\nTroubleshoot performance issues across Spark jobs, SQL workloads, pipelines, and Databricks environments.\n\nWork with structured and semi-structured datasets from multiple enterprise sources.\n\nCollaborate with Data Engineering, Analytics, AI/ML, Product, and business teams.\n\nTranslate business and analytical requirements into scalable data solutions.\n\nParticipate in technical design, code reviews, testing, deployment, and production support.\n\nMaintain engineering standards, documentation, and data-development best practices.\n\nParticipate in Agile/Scrum development activities using Jira and Confluence.\n\nMust-Have Skills\n\nCandidates should demonstr","datePosted":"2026-09-26T09:00:11.806Z","dateModified":"2026-09-26T09:00:11.806Z","hiringOrganization":{"@type":"Organization","name":"Innova Software Services","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"daacdb2f786503da84da035d"},"url":"https://jobsearcher.com/jobs/daacdb2f786503da84da035d"}}