AI Linguist

Role: AI LinguistLocation: Mountain View, CA (3 Days hybrid)Duration: 12 MonthsWe are seeking AI Linguists to join our diverse and interdisciplinary team to focus on AI quality, evaluation, and annotation and to contribute with their expertise to the annotation/human judgment pipeline development that lay at the foundation of the Client's AI model development and improvement.ResponsibilitiesDesign and scale AI quality, evaluation, and annotation workflows for models and agentic productsBuild human-in-the-loop and AI-assisted judgment systems using clear rubrics, decision rules, and quality controlsDefine measurable criteria for model behavior, including search relevance, task success, and agent performanceRun annotation and evaluation workflows across internal and vendor teams, including sampling, QA, adjudication, and calibrationConduct error analysis and experiments to improve prompts, datasets, rubrics, and workflow efficiencyPartner cross-functionally to align on quality goals, tooling, and continuous product improvementDocument processes, standardize workflows, and report evaluation results to improve quality, efficiency, and scaleBasic QualificationsB.S./B.A Degree in Computational Linguistics, Language Technologies, Linguistics, Speech Science or a related field.2+ years relevant industry experience.2+ years experience with large-scale, end-to-end human judgment flows and tools for the training/evaluation of AI models2+ years experience to breakdown linguistic data requirements into concise, clear instructions and designs1+ years experience programming in PythonPreferred QualificationsMS or Ph.D. in Computational Linguistics, Language Technologies, Linguistics or a related field.Previous experience with agentic products is a strong plusWillingness to learn technical concepts and new tools and a keen interest in technologyExperience with human judgment projects and flowsExperience with complex and scalable data annotations and evaluationsExperience with programming languages in data annotation spaceExcellent comprehension and written communication skills in EnglishSuggested SkillsAI Evaluation & Annotation DesignLinguistic Data AnalysisHuman in the Loop Workflow Development#J-18808-Ljbffr