JOBSEARCHER

Founding Engineer, Applied AI

LitmusMillbrae, CAL6 LeadOctober 26th, 2026
About Litmus Enterprises are rapidly deploying agents to automate complex work, and every one of them needs a reliable way to simulate, verify, and optimize those agents before trusting them with real work. Litmus gives them that confidence. We learn how a company's people actually work, generate the sandboxes, tasks, and verifiers to match, and evaluate its agents before they ship and once they're in production. The goal is an autonomous research engine that gives every enterprise the kind of internal benchmarking function the frontier labs have. Litmus was founded by Sidharth Srinivasan, who led AI evals at Box after its CEO recruited him out of Stanford CS. He built the function from inception, running a research engineering team and owning Box's relationships with the frontier labs and data vendors. Litmus has raised $5M from Pear, BoxGroup, Workbench, and Emergence, alongside angels including Aaron Levie (CEO of Box), Arash Ferdowsi (co-founder of Dropbox), and senior leaders at OpenAI, Anthropic, Google, and Mercor. Demand already spans Fortune 15 enterprises, top US banks, AI transformation firms, and AI-native services companies, and we're hiring to keep up. How we work: in person in FiDi, five days a week. Small team, high ownership, and we care about outcomes, not hours.
The role: Define what "good" means for agents doing real work, and prove it.
The frontier labs know how to measure and improve models for knowledge work: mock the tools, generate tasks from real usage, build verifiers, and audit the results. You'll bring that same rigor to every enterprise's agents, in finance, healthcare, legal, and enterprise operations, where the answer is rarely a simple right or wrong. You'll own the science at the core of Litmus and turn it into something that runs on its own. What you'll work on: Verifiers and graders for knowledge work, from calibrated LLM judges to checks grounded in what the agent actually did Simulated users, environments, and tasks faithful enough that passing them predicts production Fidelity, judge calibration, and coverage: knowing when you've tested enough, and when a score means something Golden trajectories, failure taxonomies, and the loop from "here's what broke" to "here's the fix" Turning what works for one customer into something the product does automatically Why it's exciting: You'll write the playbook for measuring agents doing real work, on real agents in high-stakes domains, not leaderboards. The environments and verifiers that test an agent are exactly what it takes to train one. Measurement is the first step toward agents that improve on their own. You'll work directly with customers' teams and watch your work change what they ship. You might be a fit if You've built evals, benchmarks, verifiers, reward models, RL environments, or synthetic or human data pipelines for LLMs or agents, with your own hands You have a researcher's skepticism about whether a number means anything, and an engineer's instinct to ship code to find out You've chosen risk before: founder, founding engineer, early employee, or a small, high-output team You typically have 2–6 years of experience
Nice to have: Non-code domains like legal, finance, healthcare, or enterprise operations
Post-training (SFT, RLHF, RLVR) Statistics for evaluation, benchmark design, or published eval research Probably not a fit Eval dashboards or observability tooling without hands-on eval design Model training with little interest in evaluation
What we offer: Founding equity and competitive cash
One of the first hires, working directly with the founder and customers from day one A say in how agent quality gets measured across the industry A beautiful office in FiDi Breakfast, lunch, and dinner Medical, dental, and vision coverage Equinox membership or a fitness stipend Essentially unlimited budget for AI tools and tokens Relocation support if you're moving to San Francisco