{"schemaVersion":"jobsearcher.job.v1","id":"5b231c9f80184b35a6dbb607","url":"https://jobsearcher.com/jobs/5b231c9f80184b35a6dbb607","canonicalUrl":"https://jobsearcher.com/jobs/5b231c9f80184b35a6dbb607","title":"Founding Engineer, Applied AI","description":"About Litmus\n\nEnterprises are rapidly deploying agents to automate complex work, and every one of them needs a reliable way to simulate, verify, and optimize those agents before trusting them with real work. Litmus gives them that confidence. We learn how a company's people actually work, generate the sandboxes, tasks, and verifiers to match, and evaluate its agents before they ship and once they're in production. The goal is an autonomous research engine that gives every enterprise the kind of internal benchmarking function the frontier labs have.\n\nLitmus was founded by Sidharth Srinivasan, who led AI evals at Box after its CEO recruited him out of Stanford CS. He built the function from inception, running a research engineering team and owning Box's relationships with the frontier labs and data vendors. Litmus has raised $5M from Pear, BoxGroup, Workbench, and Emergence, alongside angels including Aaron Levie (CEO of Box), Arash Ferdowsi (co-founder of Dropbox), and senior leaders at OpenAI, Anthropic, Google, and Mercor. Demand already spans Fortune 15 enterprises, top US banks, AI transformation firms, and AI-native services companies, and we're hiring to keep up.\n\nHow we work: in person in FiDi, five days a week. Small team, high ownership, and we care about outcomes, not hours.\n\nThe role\n\nDefine what \"good\" means for agents doing real work, and prove it.\n\nThe frontier labs know how to measure and improve models for knowledge work: mock the tools, generate tasks from real usage, build verifiers, and audit the results. You'll bring that same rigor to every enterprise's agents, in finance, healthcare, legal, and enterprise operations, where the answer is rarely a simple right or wrong. You'll own the science at the core of Litmus and turn it into something that runs on its own.\n\nWhat you'll work on:\n\nVerifiers and graders for knowledge work, from calibrated LLM judges to checks grounded in what the agent actually did\n\nSimulated users, environments, and tasks faithful enough that passing them predicts production\n\nFidelity, judge calibration, and coverage: knowing when you've tested enough, and when a score means something\n\nGolden trajectories, failure taxonomies, and the loop from \"here's what broke\" to \"here's the fix\"\n\nTurning what works for one customer into something the product does automatically\n\nWhy it's exciting:\n\nYou'll write the playbook for measuring agents doing real work, on real agents in high-stakes domains, not leaderboards.\n\nThe environments and verifiers that test an agent are exactly what it takes to train one. Measurement is the first step toward agents that improve on their own.\n\nYou'll work directly with customers' teams and watch your work change what they ship.\n\nYou might be a fit if\n\nYou've built evals, benchmarks, verifiers, reward models, RL environments, or synthetic or human data pipelines for LLMs or agents, with your own hands\n\nYou have a researcher's skepticism about whether a number means anything, and an engineer's instinct to ship code to find out\n\nYou've chosen risk before: founder, founding engineer, early employee, or a small, high-output team\n\nYou typically have 2–6 years of experience\n\nNice to have\n\nNon-code domains like legal, finance, healthcare, or enterprise operations\n\nPost-training (SFT, RLHF, RLVR)\n\nStatistics for evaluation, benchmark design, or published eval research\n\nProbably not a fit\n\nEval dashboards or observability tooling without hands-on eval design\n\nModel training with little interest in evaluation\n\nWhat we offer\n\nFounding equity and competitive cash\n\nOne of the first hires, working directly with the founder and customers from day one\n\nA say in how agent quality gets measured across the industry\n\nA beautiful office in FiDi\n\nBreakfast, lunch, and dinner\n\nMedical, dental, and vision coverage\n\nEquinox membership or a fitness stipend\n\nEssentially unlimited budget for AI tools and tokens\n\nRelocation support if you're moving to San Francisco","company":"Litmus","rawCompany":"litmus","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-10-26T09:29:04.401Z","occupations":[{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"17-2199.00","title":"Engineers, All Other","slug":"engineers-all-other"}],"industries":[{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"541690","title":"Other Scientific and Technical Consulting Services","slug":"other-scientific-and-technical-consulting-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Founding Engineer, Applied AI","description":"About Litmus\n\nEnterprises are rapidly deploying agents to automate complex work, and every one of them needs a reliable way to simulate, verify, and optimize those agents before trusting them with real work. Litmus gives them that confidence. We learn how a company's people actually work, generate the sandboxes, tasks, and verifiers to match, and evaluate its agents before they ship and once they're in production. The goal is an autonomous research engine that gives every enterprise the kind of internal benchmarking function the frontier labs have.\n\nLitmus was founded by Sidharth Srinivasan, who led AI evals at Box after its CEO recruited him out of Stanford CS. He built the function from inception, running a research engineering team and owning Box's relationships with the frontier labs and data vendors. Litmus has raised $5M from Pear, BoxGroup, Workbench, and Emergence, alongside angels including Aaron Levie (CEO of Box), Arash Ferdowsi (co-founder of Dropbox), and senior leaders at OpenAI, Anthropic, Google, and Mercor. Demand already spans Fortune 15 enterprises, top US banks, AI transformation firms, and AI-native services companies, and we're hiring to keep up.\n\nHow we work: in person in FiDi, five days a week. Small team, high ownership, and we care about outcomes, not hours.\n\nThe role\n\nDefine what \"good\" means for agents doing real work, and prove it.\n\nThe frontier labs know how to measure and improve models for knowledge work: mock the tools, generate tasks from real usage, build verifiers, and audit the results. You'll bring that same rigor to every enterprise's agents, in finance, healthcare, legal, and enterprise operations, where the answer is rarely a simple right or wrong. You'll own the science at the core of Litmus and turn it into something that runs on its own.\n\nWhat you'll work on:\n\nVerifiers and graders for knowledge work, from calibrated LLM judges to checks grounded in what the agent actually did\n\nSimulated users, environments, and tasks faithful enough that passing them predicts production\n\nFidelity, judge calibration, and coverage: knowing when you've tested enough, and when a score means something\n\nGolden trajectories, failure taxonomies, and the loop from \"here's what broke\" to \"here's the fix\"\n\nTurning what works for one customer into something the product does automatically\n\nWhy it's exciting:\n\nYou'll write the playbook for measuring agents doing real work, on real agents in high-stakes domains, not leaderboards.\n\nThe environments and verifiers that test an agent are exactly what it takes to train one. Measurement is the first step toward agents that improve on their own.\n\nYou'll work directly with customers' teams and watch your work change what they ship.\n\nYou might be a fit if\n\nYou've built evals, benchmarks, verifiers, reward models, RL environments, or synthetic or human data pipelines for LLMs or agents, with your own hands\n\nYou have a researcher's skepticism about whether a number means anything, and an engineer's instinct to ship code to find out\n\nYou've chosen risk before: founder, founding engineer, early employee, or a small, high-output team\n\nYou typically have 2–6 years of experience\n\nNice to have\n\nNon-code domains like legal, finance, healthcare, or enterprise operations\n\nPost-training (SFT, RLHF, RLVR)\n\nStatistics for evaluation, benchmark design, or published eval research\n\nProbably not a fit\n\nEval dashboards or observability tooling without hands-on eval design\n\nModel training with little interest in evaluation\n\nWhat we offer\n\nFounding equity and competitive cash\n\nOne of the first hires, working directly with the founder and customers from day one\n\nA say in how agent quality gets measured across the industry\n\nA beautiful office in FiDi\n\nBreakfast, lunch, and dinner\n\nMedical, dental, and vision coverage\n\nEquinox membership or a fitness stipend\n\nEssentially unlimited budget for AI tools and tokens\n\nRelocation support if you're moving to San Francisco","datePosted":"2026-10-26T09:29:04.401Z","dateModified":"2026-10-26T09:29:04.401Z","hiringOrganization":{"@type":"Organization","name":"Litmus","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"5b231c9f80184b35a6dbb607"},"url":"https://jobsearcher.com/jobs/5b231c9f80184b35a6dbb607"}}