Machine Learning Engineer, RL Environments - Internship

Location: San Francisco preferred, remote consideredCompensation: Paid internshipAbout UsPreference Model is building automated ML research engineering.Existing frontier models are brittle when applied to real-world ML tasks. The present bottleneck is the lack of high-quality RL training environments. Our first step is to build RL environments that reflect real-world complexity, with diverse tasks and robust reward functions.Our founding team has previous experience on Anthropic’s data team building data infrastructure, and datasets behind Claude. We are partnering with leading AI labs to push AI closer to achieving its transformative potential.About The RoleWe're looking for PhD or Master's students, and gifted undergrads to spend an internship with us working on building RL training environments for large language models.This role blends research and engineering. It will require you to both develop novel approaches and realize them in code. Your work will include designing and implementing RL environments, conducting experiments and evaluations, delivering your work into production training runs, and collaborating with other researchers and engineers.What you'll doDesign and build RL environments that test LLM reasoning on ML, systems, and research problemsWrite clean, production-grade Python (not notebooks)Work with Docker, build reproducible environments, debug when things breakTranslate ML papers and concepts into concrete training tasksWhat We are Looking For (Qualifications):You're an undergrad or PhD student in CS, ML, math, physics, or a related field. You write real code, not just research prototypes. You read ML papers for fun in your free time.Must haveStrong Python skillsFamiliarity with how LLMs work, what they're good at, and where they fall shortAbility to work independently, take feedback, and iterate fastYou may be a good fit if one of the following appliesYou understand transformer internals and have worked with training or inference codeYou've written CUDA kernels or worked with low-level GPU programmingYou have a research area you know deeply (publications, public code, or strong coursework)You read broadly across ML and can connect ideas from different subfieldsYou've built interactive environments, simulations, or complex software systemsWhat we offer:Paid Internship with opportunity to return full time based on performanceOwnership and autonomy in a fast moving startup environmentOpportunity to work with top machine learning engineersCompetitive cash and equity compensation (>90th percentile)Lunch provided everyday onsiteWeekly snack ordersNote: We utilize AI note-taking during our interview sessions to ensure we capture all answers and details accurately. Candidates are allowed to use AI note-takers as well, however, no other AI tools are permitted during any live interviews.