{"schemaVersion":"jobsearcher.job.v1","id":"fce015a35eec9dcb0c019f0d","url":"https://jobsearcher.com/jobs/fce015a35eec9dcb0c019f0d","canonicalUrl":"https://jobsearcher.com/jobs/fce015a35eec9dcb0c019f0d","title":"Spark Developer (Search Integration)","description":"Spark Developer (Search Integration)Location: Pleasanton, CAWe are looking for a Spark Developer with OpenSearch/Algolia expertise who can design, build, and optimize scalable data pipelines to ingest, transform, and index large-scale datasets into search engines for fast retrieval. Utilize Spark SQL to process data from various sources (S3, Kafka) for real-time indexing in OpenSearch or Algolia.Focus: Spark (ETL/Streaming) + Search Engines (OpenSearch/Algolia)Objective: Power real-time, relevant, and fast search experiences. Key Responsibilities:Data Pipelines: Design, develop, and maintain high-performance Spark jobs (PySpark) to process, transform, and clean large datasets.Index Management: Ingest data into OpenSearch or Algolia, optimizing index strategy, mapping, and document structuring for maximum search efficiency.Optimization: Tune Spark applications (data partitioning, caching, shuffle tuning) and search engines (query performance, indexing speed).Streaming/Batch: Implement both batch ETL jobs and real-time streaming solutions (Spark Streaming/Kafka) to keep search indexes updated.Collaboration: Work with backend teams to integrate search functionality into applications and debug search relevance issues.Required Skills and Qualifications:Core Spark: Strong experience with Apache Spark RDD/DataFrame APIs, PySpark.Search Tech: Experience in indexing, querying, and managing clusters in OpenSearch (formerly Elasticsearch) or Algolia.","company":"Rbm Software","rawCompany":"rbm software","city":"Pleasanton","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-04-12T19:22:38.038Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"},{"code":"15-2051.00","title":"Data Scientists","slug":"data-scientists"}],"industries":[{"code":"519290","title":"Web Search Portals and All Other Information Services","slug":"web-search-portals-and-all-other-information-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Spark Developer (Search Integration)","description":"Spark Developer (Search Integration)Location: Pleasanton, CAWe are looking for a Spark Developer with OpenSearch/Algolia expertise who can design, build, and optimize scalable data pipelines to ingest, transform, and index large-scale datasets into search engines for fast retrieval. Utilize Spark SQL to process data from various sources (S3, Kafka) for real-time indexing in OpenSearch or Algolia.Focus: Spark (ETL/Streaming) + Search Engines (OpenSearch/Algolia)Objective: Power real-time, relevant, and fast search experiences. Key Responsibilities:Data Pipelines: Design, develop, and maintain high-performance Spark jobs (PySpark) to process, transform, and clean large datasets.Index Management: Ingest data into OpenSearch or Algolia, optimizing index strategy, mapping, and document structuring for maximum search efficiency.Optimization: Tune Spark applications (data partitioning, caching, shuffle tuning) and search engines (query performance, indexing speed).Streaming/Batch: Implement both batch ETL jobs and real-time streaming solutions (Spark Streaming/Kafka) to keep search indexes updated.Collaboration: Work with backend teams to integrate search functionality into applications and debug search relevance issues.Required Skills and Qualifications:Core Spark: Strong experience with Apache Spark RDD/DataFrame APIs, PySpark.Search Tech: Experience in indexing, querying, and managing clusters in OpenSearch (formerly Elasticsearch) or Algolia.","datePosted":"2026-04-12T19:22:38.038Z","dateModified":"2026-04-12T19:22:38.038Z","hiringOrganization":{"@type":"Organization","name":"Rbm Software","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Pleasanton","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"fce015a35eec9dcb0c019f0d"},"url":"https://jobsearcher.com/jobs/fce015a35eec9dcb0c019f0d"}}