{"schemaVersion":"jobsearcher.job.v1","id":"c7b8d2ebde57c360931781b0","url":"https://jobsearcher.com/jobs/c7b8d2ebde57c360931781b0","canonicalUrl":"https://jobsearcher.com/jobs/c7b8d2ebde57c360931781b0","title":"Principal Data Processing Engineer - OSS Engineering Mountain View, CA","description":"Principal Data Processing Engineer - OSS\r\nMountain View, CA\r\nAbout DataPelago:DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray, and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for world-class engineers to join our team and shape the future of accelerated data processing.\r\nThe Role:As a Principal Data Processing Engineer (OSS), you will be a key individual contributor in adopting and advancing the capabilities of open-source software (OSS) platforms such as Apache Gluten, Velox, Apache Spark, and Apache Flink in the context of DataPelago's data processing engine. You will enhance the functional breadth, performance, scale, and reliability of the DataPelago engine through downstream and upstream contributions. You will have the opportunity to engage with the community working on these platforms. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.\r\nWhat You'll Do:Influence the architecture of how our data processing engine interfaces with open-source platforms and engines.\r\nLead the design of functional and performance enhancements to open source platforms such as Apache Gluten and Velox, and their integration with our data processing engine.\r\nIndividually design, implement, test, optimize, and maintain components of the data processing engine.\r\nAnalyze the technology roadmap of Apache Gluten, Velox, and equivalent platforms and identify opportunities for our engine to enhance technology and product leadership.\r\nCollaboration: Partner with engineering, product management, the open-source community and customer success teams.\r\nFoster best practices in design and code reviews, testing, CI/CD, and issue resolution to maintain the highest product quality, security, efficiency, and productivity.\r\nWhat You'll Bring:BS/MS in Computer Science (or a related field) with 6+ years of relevant experience\r\n3+ years of deep technical experience in instrumenting, analyzing, and optimizing the performance of data processing engine components on benchmark and customer workloads.\r\nSound knowledge of the architecture and internal operation of one or more of Apache Spark, Apache Flink, Presto/Trino.\r\nDemonstrated experience in the design, development, and successful release of high-performance data processing engines for large production deployments.\r\nExceptional programming skills in C, C++, and Java.\r\nExtensive development experience in Linux environments.\r\nExcellent communication and collaboration skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.\r\nStrong analytical and problem-solving skills with a passion for performance optimization.\r\nLocation Considerations:We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and at remote locations.\r\nWhy Join DataPelago?Technical Leadership: Take a leadership role in shaping the architecture and development of how our core engine works with open source data processing platforms\r\nCutting-Edge Innovation: Work on challenging problems at the forefront of accelerated computing and data processing.\r\nSignificant Impact: Your contributions will directly impact the performance and scalability of our mission-critical platform.\r\nMentorship and Growth: Mentor and guide other talented engineers while expanding your own technical expertise.\r\nCompetitive compensation, stock options, comprehensive benefits package, and leadership development opportunities#J-18808-Ljbffr","company":"Datapelago","rawCompany":"datapelago","city":"Mountain View","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-10-04T03:05:45.971Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1243.01","title":"Data Warehousing Specialists","slug":"data-warehousing-specialists"}],"industries":[{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Principal Data Processing Engineer - OSS Engineering Mountain View, CA","description":"Principal Data Processing Engineer - OSS\r\nMountain View, CA\r\nAbout DataPelago:DataPelago is at the forefront of revolutionizing data processing for traditional analytics and cutting-edge GenAI preprocessing. We are building an innovative data processing engine that is transforming how Apache Spark, Apache Flink, Ray, and others operate on diverse, large-scale data. Our team of engineers drive and adopt advances in hardware-accelerated computing, parallel processing of large-scale data, query optimization, distributed systems, compilers, machine learning, and cloud-native computing. We are looking for world-class engineers to join our team and shape the future of accelerated data processing.\r\nThe Role:As a Principal Data Processing Engineer (OSS), you will be a key individual contributor in adopting and advancing the capabilities of open-source software (OSS) platforms such as Apache Gluten, Velox, Apache Spark, and Apache Flink in the context of DataPelago's data processing engine. You will enhance the functional breadth, performance, scale, and reliability of the DataPelago engine through downstream and upstream contributions. You will have the opportunity to engage with the community working on these platforms. This is a unique opportunity to make a significant impact on a category-defining product and work with a talented team of engineers.\r\nWhat You'll Do:Influence the architecture of how our data processing engine interfaces with open-source platforms and engines.\r\nLead the design of functional and performance enhancements to open source platforms such as Apache Gluten and Velox, and their integration with our data processing engine.\r\nIndividually design, implement, test, optimize, and maintain components of the data processing engine.\r\nAnalyze the technology roadmap of Apache Gluten, Velox, and equivalent platforms and identify opportunities for our engine to enhance technology and product leadership.\r\nCollaboration: Partner with engineering, product management, the open-source community and customer success teams.\r\nFoster best practices in design and code reviews, testing, CI/CD, and issue resolution to maintain the highest product quality, security, efficiency, and productivity.\r\nWhat You'll Bring:BS/MS in Computer Science (or a related field) with 6+ years of relevant experience\r\n3+ years of deep technical experience in instrumenting, analyzing, and optimizing the performance of data processing engine components on benchmark and customer workloads.\r\nSound knowledge of the architecture and internal operation of one or more of Apache Spark, Apache Flink, Presto/Trino.\r\nDemonstrated experience in the design, development, and successful release of high-performance data processing engines for large production deployments.\r\nExceptional programming skills in C, C++, and Java.\r\nExtensive development experience in Linux environments.\r\nExcellent communication and collaboration skills, with the ability to articulate complex technical concepts to both technical and non-technical audiences.\r\nStrong analytical and problem-solving skills with a passion for performance optimization.\r\nLocation Considerations:We value face-to-face collaboration, but recognize that talent can be found anywhere. Our engineering team works at our headquarters in Mountain View, CA, at our India office in Hyderabad, and at remote locations.\r\nWhy Join DataPelago?Technical Leadership: Take a leadership role in shaping the architecture and development of how our core engine works with open source data processing platforms\r\nCutting-Edge Innovation: Work on challenging problems at the forefront of accelerated computing and data processing.\r\nSignificant Impact: Your contributions will directly impact the performance and scalability of our mission-critical platform.\r\nMentorship and Growth: Mentor and guide other talented engineers while expanding your own technical expertise.\r\nCompetitive compensation, stock options, comprehensive benefits package, and leadership development opportunities#J-18808-Ljbffr","datePosted":"2026-10-04T03:05:45.971Z","dateModified":"2026-10-04T03:05:45.971Z","hiringOrganization":{"@type":"Organization","name":"Datapelago","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Mountain View","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"c7b8d2ebde57c360931781b0"},"url":"https://jobsearcher.com/jobs/c7b8d2ebde57c360931781b0"}}