{"schemaVersion":"jobsearcher.job.v1","id":"37483ac48da148301efce50b","url":"https://jobsearcher.com/jobs/37483ac48da148301efce50b","canonicalUrl":"https://jobsearcher.com/jobs/37483ac48da148301efce50b","title":"Evaluation Lead","description":"About Archetype AI\nArchetype AI is developing the world's first AI platform to bring AI into the real world. Formed by an exceptionally high-caliber team from Google, Archetype AI is building a foundation model for the physical world, a real‑time multimodal LLM for real life, transforming real‑world data into valuable insights and knowledge that people will be able to interact with naturally. It will help people in their real lives, not just online, because it understands the real‑time physical environment and everything that happens in it.\n\nSupported by deep tech venture funds in Silicon Valley, Archetype AI is currently at the Series A stage and is progressing rapidly to develop technology for its next stage. This presents a unique and once‑in‑a‑lifetime opportunity to be part of an exciting AI team at the beginning of their journey, located in the heart of Silicon Valley. Our team is headquartered in San Mateo, California, with team members throughout the US and Europe. We are actively growing, so if you are an exceptional candidate excited to work on the cutting edge of physical AI and don’t see a role that exactly fits you below you can contact us directly with your resume via jobsarchetypeaiio.\n\nAbout The Role\nArchetype AI is seeking a hands‑on Evaluation Lead to build and assess model performance for physical AI. You will design and implement advanced evaluation techniques for assessing the strengths and weaknesses of real‑world AI models, and build and scale evaluation frameworks to rapidly test and generate reports on model performance. Responsibilities include partnering closely with research and engineering teams to develop evaluation methodologies, analytically assessing and improving test datasets, uncovering model weaknesses or risks, and tracking competitive industry benchmarks. This is a high‑impact role for someone who thrives in a fast‑paced AI environment and wants to directly influence our path as we scale our AI technologies and business.\n\nCore Responsibilities\n\nDrive Benchmarking & Evaluation\n\nDesign and implement rigorous evaluation methodologies and benchmarks for measuring model effectiveness, reliability, alignment, and safety\n\nLead evaluation of model performance, ranging from offline experiments to full production model testing\n\nBuild & Scale Evaluation Frameworks\n\nDesign and oversee the pipelines, dashboards, and tools that automate model evaluation\n\nDesign and oversee tools for A/B model testing, regression testing, and production model performance\n\nLead Evaluation Strategy\n\nDevelop and implement strategies for evaluating physical AI models that can scale across a broad range of real‑world use cases, sensor types, and edge cases\n\nPlan, run, and oversee evaluations, across internal teams and external customers\n\nDrive edge case discovery, red‑teaming, safety, privacy, and risk evaluation – feeding back knowledge to key stakeholders in research and engineering teams\n\nKey Requirements\n\nExtensive expertise in evaluating AI and machine learning models, ideally in physical AI or a related AI field\n\nExperience in designing, implementing, and refining evaluation metrics\n\nDeep understanding of machine learning, AI, and generative models\n\nExcellent Python and software engineering skills\n\nExperience designing and building scalable data pipelines and evaluation tools\n\nExperience collaborating closely with key stakeholders from research, engineering, and product teams\n\nStrong communication and documentation skills, with a bias for creating detailed evaluation reports that help drive model performance\n\nStartup‑ready mindset with the ability to thrive in high‑velocity, high‑ambiguity environments\n\nMinimum Qualifications\n\nExtensive expertise in evaluating AI and machine learning models, ideally in physical AI or a related AI field\n\nExperience in designing, implementing, and refining evaluation metrics\n\nDeep understanding of machine learning, AI, and generative models\n\nExcellent Python and software engineering skills\n\nExperience designing and building scalable data pipelines and evaluation tools\n\nExperience collaborating closely with key stakeholders from research, engineering, and product teams\n\nStrong communication and documentation skills, with a bias for creating detailed evaluation reports that help drive model performance\n\nStartup‑ready mindset with the ability to thrive in high‑velocity, high‑ambiguity environments\n\nWhat We Would Love To See\n\nExperience evaluating real‑world, real‑time algorithms\n\nExperience evaluating a broad range of sensor types, such as cameras, LIDAR, physical sensors, RF sensors, and beyond\n\nA strong scientific approach to evaluation and understanding model performance\n\nExperience in evaluating production algorithms\n\nExperience building and curating data campaigns to create extensive test datasets\n\nExperience managing internal teams and/or external vendors\n\n#J-18808-Ljbffr","company":"SupportFinity","rawCompany":"supportfinity","city":"Millbrae","state":"CA","isRemote":false,"isActive":true,"createdAt":"2026-06-20T04:20:35.963Z","occupations":[{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"},{"code":"17-2112.02","title":"Validation Engineers","slug":"validation-engineers"},{"code":"11-9121.00","title":"Natural Sciences Managers","slug":"natural-sciences-managers"}],"industries":[{"code":"541690","title":"Other Scientific and Technical Consulting Services","slug":"other-scientific-and-technical-consulting-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541715","title":"Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology)","slug":"research-and-development-in-the-physical-engineering-and-life-sciences-except-nanotechnology-and-biotechnology"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Evaluation Lead","description":"About Archetype AI\nArchetype AI is developing the world's first AI platform to bring AI into the real world. Formed by an exceptionally high-caliber team from Google, Archetype AI is building a foundation model for the physical world, a real‑time multimodal LLM for real life, transforming real‑world data into valuable insights and knowledge that people will be able to interact with naturally. It will help people in their real lives, not just online, because it understands the real‑time physical environment and everything that happens in it.\n\nSupported by deep tech venture funds in Silicon Valley, Archetype AI is currently at the Series A stage and is progressing rapidly to develop technology for its next stage. This presents a unique and once‑in‑a‑lifetime opportunity to be part of an exciting AI team at the beginning of their journey, located in the heart of Silicon Valley. Our team is headquartered in San Mateo, California, with team members throughout the US and Europe. We are actively growing, so if you are an exceptional candidate excited to work on the cutting edge of physical AI and don’t see a role that exactly fits you below you can contact us directly with your resume via jobsarchetypeaiio.\n\nAbout The Role\nArchetype AI is seeking a hands‑on Evaluation Lead to build and assess model performance for physical AI. You will design and implement advanced evaluation techniques for assessing the strengths and weaknesses of real‑world AI models, and build and scale evaluation frameworks to rapidly test and generate reports on model performance. Responsibilities include partnering closely with research and engineering teams to develop evaluation methodologies, analytically assessing and improving test datasets, uncovering model weaknesses or risks, and tracking competitive industry benchmarks. This is a high‑impact role for someone who thrives in a fast‑paced AI environment and wants to directly influence our path as we scale our AI technologies and business.\n\nCore Responsibilities\n\nDrive Benchmarking & Evaluation\n\nDesign and implement rigorous evaluation methodologies and benchmarks for measuring model effectiveness, reliability, alignment, and safety\n\nLead evaluation of model performance, ranging from offline experiments to full production model testing\n\nBuild & Scale Evaluation Frameworks\n\nDesign and oversee the pipelines, dashboards, and tools that automate model evaluation\n\nDesign and oversee tools for A/B model testing, regression testing, and production model performance\n\nLead Evaluation Strategy\n\nDevelop and implement strategies for evaluating physical AI models that can scale across a broad range of real‑world use cases, sensor types, and edge cases\n\nPlan, run, and oversee evaluations, across internal teams and external customers\n\nDrive edge case discovery, red‑teaming, safety, privacy, and risk evaluation – feeding back knowledge to key stakeholders in research and engineering teams\n\nKey Requirements\n\nExtensive expertise in evaluating AI and machine learning models, ideally in physical AI or a related AI field\n\nExperience in designing, implementing, and refining evaluation metrics\n\nDeep understanding of machine learning, AI, and generative models\n\nExcellent Python and software engineering skills\n\nExperience designing and building scalable data pipelines and evaluation tools\n\nExperience collaborating closely with key stakeholders from research, engineering, and product teams\n\nStrong communication and documentation skills, with a bias for creating detailed evaluation reports that help drive model performance\n\nStartup‑ready mindset with the ability to thrive in high‑velocity, high‑ambiguity environments\n\nMinimum Qualifications\n\nExtensive expertise in evaluating AI and machine learning models, ideally in physical AI or a related AI field\n\nExperience in designing, implementing, and refining evaluation metrics\n\nDeep understanding of machine learning, AI, and generative models\n\nExcellent Python and software engineering skills\n\nExperience designing and building scalable data pipelines and evaluation tools\n\nExperience collaborating closely with key stakeholders from research, engineering, and product teams\n\nStrong communication and documentation skills, with a bias for creating detailed evaluation reports that help drive model performance\n\nStartup‑ready mindset with the ability to thrive in high‑velocity, high‑ambiguity environments\n\nWhat We Would Love To See\n\nExperience evaluating real‑world, real‑time algorithms\n\nExperience evaluating a broad range of sensor types, such as cameras, LIDAR, physical sensors, RF sensors, and beyond\n\nA strong scientific approach to evaluation and understanding model performance\n\nExperience in evaluating production algorithms\n\nExperience building and curating data campaigns to create extensive test datasets\n\nExperience managing internal teams and/or external vendors\n\n#J-18808-Ljbffr","datePosted":"2026-06-20T04:20:35.963Z","dateModified":"2026-06-20T04:20:35.963Z","hiringOrganization":{"@type":"Organization","name":"SupportFinity","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"37483ac48da148301efce50b"},"url":"https://jobsearcher.com/jobs/37483ac48da148301efce50b"}}