Senior Software Engineer, AI Coding Evaluation (Contract)
We're looking for experienced software engineers to build one AI coding evaluation task each, from a codebase they know well. You pick a real change in a repository you know from the inside, one that a leading AI coding model gets wrong. You write the fix, the tests that prove it, and a short grading rubric. The task is used to measure and train AI coding models. Remote, any location. 15 to 20 hours. Due Tuesday 29 September. $80 to $100 per hour (USD), up to 20 hours. What you'll deliver A one-page proposal, approved by us before you build The repo building and running its tests offline in Docker, at a pinned commit Your own fix, written to the standard a maintainer would merge Tests that reject a nearly-right fix3 to 5 review criteria for style, scope and design
Requirements:
- Ideally 3+ years of professional software engineering
- Code merged into a public open-source repository, ideally one you know well. A private codebase you own also works (not employer or client code)
- Strong in at least one of Python, TypeScript, Go, Rust, Java or C++Solid Docker, Linux, Git and testing skills
- Regular use of an AI coding tool
- Experience writing tests, rubrics or evaluation criteria for AI models
- A secure personal or work computer (no public or shared machines)
- Available now for 15 to 20 hours
Nice to have:
Deep expertise in one area, such as databases, compilers, distributed systems, security or infrastructure- Experience building agent tasks or RL environments Terms- Paid only for a task that passes our quality review- You sign Askable's contract and accept its terms- The work is used for AI training. The change and your solution must be your own, and you must be free to license them to us- Don't publish the work anywhere: no public pull request or public fork Apply at https://www.askable.com/experts/roles/senior-software-engineer