Strategic Project Lead

Who we are and what we doContra Labs is a human-centered research lab focused on understanding how creatives use AI, how AI tools can better align with creative needs, and how to preserve the diversity and originality that defines great creative work in an era of increasing homogenization.We connect a global network of top creative talent with frontier AI labs and product companies. By grounding our research in real creative workflows, we help define what "good" looks like across design, video, imagery, and beyond. Our work powers benchmarking, evaluation, and post-training for leading AI models and applications.Built on top of Contra, the professional network for independent creatives, Contra Labs turns human judgment into the training and evaluation layer that helps AI tools better serve the people who use them. Creatives are at the center of everything we do.Why this role existsContra Labs is scaling across client engagements including post-training dataset creation, benchmarking programs, and ongoing evaluation partnerships. We need someone who can own the research and delivery side of this work end-to-end. Projects span AI labs, creative tool companies, and research teams, each with different requirements, timelines, and quality bars.This person scopes the methodology, designs the evaluation protocols, spins up the right evaluators, manages quality throughout, and makes sure we deliver rigorous, actionable results.What you'll doDesign and lead research studies including annotation protocols, rubrics, and evaluation methodologies for creative and multimodal outputsConduct and oversee qualitative and quantitative analysis, synthesizing findings into actionable insights and recommendations for clientsInterface with creative domain experts, manage evaluator teams, and calibrate for consistent quality through real-time course-correctionDevelop and standardize annotation, data quality, and inter-rater reliability protocols across projectsCommunicate research findings effectively to stakeholders through reports, presentations, and workshopsCollaborate with engineering on tooling needs and contribute to Contra Labs' positioning through case studies and benchmark publicationsTranslate client goals and priorities into well-scoped research plans, managing multiple concurrent projects with different timelines and requirementsAbout youYou have deep experience in research methods, both qualitative and quantitative. You know how to design a study, run it, analyze the data, and translate findings into clear, actionable directionsYou have taste. You understand creative quality intuitively and can build evaluation frameworks that capture itYou are more interested in people than in technology for its own sake. You understand how humans judge creative work and can design systems that capture that signal faithfullyYou can context-switch across projects and manage priorities in a fast-moving environment. Design study thinking comes naturally to youHigh ownership, low ego, bias toward action. You ship v1, iterate, and do whatever it takes to deliverExperienceMaster's degree or PhD in HCI, cognitive science, psychology, design research, or a related field (or equivalent depth of industry experience)3+ years of hands-on experience conducting research: designing studies, running evaluations, analyzing qualitative and quantitative data, and presenting findingsExperience with statistical methods including inter-rater reliability, sampling, regression analysis, and significance testingProficiency with data analysis tools (Python, SQL) for working with evaluation data at scaleExperience in data labeling, annotation, human-in-the-loop systems, or AI/ML evaluation workflowsTrack record managing multiple concurrent projects, including annotation, evaluation, or QA workflowsStrong understanding of creative domains and what "good" looks likeBonusExperience with RLHF, LLM evaluation, benchmark design, or human-in-the-loop systemsBackground in ethnography, UX research, or design study methodologiesStartup or 0-to-1 experienceRequirementsBased in NYC, USAIn-office 5 days/week (Williamsburg)Total CompSalary: $150,000-$200,000 USD + EquityMedical, Dental, Vision Benefits401k MatchingWe will provide you with a company laptop on your start dateInterview ProcessInterview with Recruiting Team (20 minutes)Interview with CEO & Co-Founder (30 minutes)Technical Interview with Research/Methods Focus (45 minutes)Interview with VP of Product (45 minutes)Paid Case Study and Presentation (60 minutes)Note: Contra communicates with applicants through @contra.com domains only. We never ask for money from potential employees. For the latest job postings, visit Contra Careers.