{"schemaVersion":"jobsearcher.job.v1","id":"a76f335da68b73db29f2fe71","url":"https://jobsearcher.com/jobs/a76f335da68b73db29f2fe71","canonicalUrl":"https://jobsearcher.com/jobs/a76f335da68b73db29f2fe71","title":"Research Engineer","description":"Research Engineer, Coding Agent Optimization Engine\nSan Francisco, in person · Full-time · Entelligence\nCoding agents ship more code every week, and they fail in repeatable ways: the same wrong assumption, the same missed test, the same review comment on PR after PR. Our Coding Agent Optimization Engine closes that loop. It mines agent sessions and PR review comments for failures, traces them to where the harness went wrong, and feeds the lessons back so the next session does not repeat them. It also routes every turn to the most efficient model that can solve it. It runs on live traffic from Claude Code, Codex, Cursor and API clients. We want a research engineer to make it measurably better at both.\nKey Focus Areas\n\nFailure mining. Captures coding-agent sessions and links them to the PRs they produced, then mines transcripts and review comments for errors, struggles and recurring root causes.\nLearning loop. Turns review findings into root-cause patterns and tested rules, and delivers them to the next agent session as guidance cards through CLI hooks.\nPre-PR judge. A reviewer model checks an agent's change before the PR is opened, and the harness reports back what happened in each session.\nCost-aware routing. Picks a difficulty tier per instruction turn and weights models by expected cost per solved task, with prompt-cache affinity, failover and per-org constraints.\n\nWhat you will work on\n\nFailure attribution. Given a session transcript and the review comments on its PR, find where the harness went wrong: missing context, a bad tool loop, a skipped check, the wrong model.\nPrevention that measurably works. Decide which lessons become rules, cards or judge checks, and prove with held-out data that they cut repeat findings without flooding agents with noise.\nEvaluation. Own the benchmark harness of record: replay real agentic tasks headlessly against competing harness configs and routing policies, and measure quality vs. cost honestly.\nEngineering world model. Build the comprehensive data layer connecting sessions, PRs, projects, errors and incidents. Leverage the world model to catch issues from happening in future sessions.\nCost and outcome modeling. Turn measured usage, cache shares, latency and review outcomes into continuously refreshed model and harness parameters.\n\nYour Background\n\n3+ years building ML or LLM systems that run in production, or equivalent research experience with shipped code.\nStrong Python and comfort reading a large production codebase.\nExperiment design you can defend: held-out sets, variance, confidence intervals, leakage checks.\nHands-on experience with LLM APIs, agents or tool-use loops, and their failure modes.\nWork on agent evaluation, failure analysis, LLM-as-judge or automated code review.\nWork on routing, cascades, bandits or learning-to-defer.\nExperience with LLM inference economics: prompt caching, batching, token pricing.\nBuilt or maintained public evals (SWE-bench, Terminal-Bench style harnesses).\nPublished research or well-known open-source work in ML systems or developer tools.\n\nCompensation and benefits\nBase salary: $170,000 - $210,000 plus meaningful equity. Final offer depends on level and experience. Healthcare and a generous compute budget for experiments.\n#J-18808-Ljbffr","company":"Entelligence","rawCompany":"entelligence","city":"Millbrae","state":"CA","isRemote":false,"isActive":false,"createdAt":"2026-11-16T03:58:33.760Z","occupations":[{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"17-2199.00","title":"Engineers, All Other","slug":"engineers-all-other"}],"industries":[{"code":"541715","title":"Research and Development in the Physical, Engineering, and Life Sciences (except Nanotechnology and Biotechnology)","slug":"research-and-development-in-the-physical-engineering-and-life-sciences-except-nanotechnology-and-biotechnology"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541720","title":"Research and Development in the Social Sciences and Humanities","slug":"research-and-development-in-the-social-sciences-and-humanities"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Research Engineer","description":"Research Engineer, Coding Agent Optimization Engine\nSan Francisco, in person · Full-time · Entelligence\nCoding agents ship more code every week, and they fail in repeatable ways: the same wrong assumption, the same missed test, the same review comment on PR after PR. Our Coding Agent Optimization Engine closes that loop. It mines agent sessions and PR review comments for failures, traces them to where the harness went wrong, and feeds the lessons back so the next session does not repeat them. It also routes every turn to the most efficient model that can solve it. It runs on live traffic from Claude Code, Codex, Cursor and API clients. We want a research engineer to make it measurably better at both.\nKey Focus Areas\n\nFailure mining. Captures coding-agent sessions and links them to the PRs they produced, then mines transcripts and review comments for errors, struggles and recurring root causes.\nLearning loop. Turns review findings into root-cause patterns and tested rules, and delivers them to the next agent session as guidance cards through CLI hooks.\nPre-PR judge. A reviewer model checks an agent's change before the PR is opened, and the harness reports back what happened in each session.\nCost-aware routing. Picks a difficulty tier per instruction turn and weights models by expected cost per solved task, with prompt-cache affinity, failover and per-org constraints.\n\nWhat you will work on\n\nFailure attribution. Given a session transcript and the review comments on its PR, find where the harness went wrong: missing context, a bad tool loop, a skipped check, the wrong model.\nPrevention that measurably works. Decide which lessons become rules, cards or judge checks, and prove with held-out data that they cut repeat findings without flooding agents with noise.\nEvaluation. Own the benchmark harness of record: replay real agentic tasks headlessly against competing harness configs and routing policies, and measure quality vs. cost honestly.\nEngineering world model. Build the comprehensive data layer connecting sessions, PRs, projects, errors and incidents. Leverage the world model to catch issues from happening in future sessions.\nCost and outcome modeling. Turn measured usage, cache shares, latency and review outcomes into continuously refreshed model and harness parameters.\n\nYour Background\n\n3+ years building ML or LLM systems that run in production, or equivalent research experience with shipped code.\nStrong Python and comfort reading a large production codebase.\nExperiment design you can defend: held-out sets, variance, confidence intervals, leakage checks.\nHands-on experience with LLM APIs, agents or tool-use loops, and their failure modes.\nWork on agent evaluation, failure analysis, LLM-as-judge or automated code review.\nWork on routing, cascades, bandits or learning-to-defer.\nExperience with LLM inference economics: prompt caching, batching, token pricing.\nBuilt or maintained public evals (SWE-bench, Terminal-Bench style harnesses).\nPublished research or well-known open-source work in ML systems or developer tools.\n\nCompensation and benefits\nBase salary: $170,000 - $210,000 plus meaningful equity. Final offer depends on level and experience. Healthcare and a generous compute budget for experiments.\n#J-18808-Ljbffr","datePosted":"2026-11-16T03:58:33.760Z","dateModified":"2026-11-16T03:58:33.760Z","hiringOrganization":{"@type":"Organization","name":"Entelligence","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Millbrae","addressRegion":"CA","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"a76f335da68b73db29f2fe71"},"url":"https://jobsearcher.com/jobs/a76f335da68b73db29f2fe71"}}