Software Engineer, RL Environments

About UsPreference Model is building automated ML research engineering.Existing frontier models are brittle when applied to real-world ML tasks. The present bottleneck is the lack of high-quality RL training environments. Our first step is to build RL environments that reflect real-world complexity, with diverse tasks and robust reward functions.Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential.About The RoleFrontier models still fail at the complex, judgment-heavy work that would make them genuinely transformative: long-horizon research, system design under constraints, iterative debugging in unfamiliar environments. The bottleneck isn't compute; it's training data. We build the RL environments that expose those failures and the infrastructure that turns them into reward.The role blends research and engineering. It will require you to both develop novel approaches and realize them in code. Your work will include designing and implementing RL environments, conducting experiments and evaluations, delivering your work into production training runs, and collaborating with other teams.What You Will DoDesign and build RL environments end-to-end: Own the full lifecycle: tasks, reward functions, grading infrastructure, failure analysis, and iteration until environments produce clean signal. Build RL training infrastructure: Develop scalable post-training systems including orchestration, performance optimization, and monitoring. Create model evaluations: Define what good agent performance looks like and build the tooling to measure it. Shape technical strategy: Drive architecture decisions and help build our engineering culture as an early team member. What We are Looking For4+ years of software engineering experience with strong project ownershipDeep expertise in at least one domain: infra, distributed systems, performance, security, or research toolingSkilled in Python, Rust, or TypeScript across the full stackHands-on experience with Kubernetes, AWS, or GCPHave extensive experience working with coding agentsThrive working independently on ambiguous, high-ownership problemsNice-to-HavesML infrastructure or RL systems experienceSimulation environments or LLM eval pipelinesDistributed systems or performance optimizationNo prior ML experience requiredThis role is not a good fit if you want a product role shipping features to end users.What we offerCompetitive cash and equity compensation (>90th percentile)Ownership and autonomy in a fast moving startup environmentOpportunity to work with top machine learning engineersHealth, vision, dental, benefits401K matchLunch provided everyday onsiteWeekly snack ordersVisa sponsorship & relocation support availableWe value diverse perspectives and experiences. If you're excited about this role but don't check every box, we still encourage you to apply.Note: We utilize AI note-taking during our interview sessions to ensure we capture all answers and details accurately. Candidates are allowed to use AI note-takers as well, however, no other AI tools are permitted during any live interviews. Compensation Range: $180K - $300K