{"schemaVersion":"jobsearcher.job.v1","id":"27d76712adf1da51d7900855","url":"https://jobsearcher.com/jobs/27d76712adf1da51d7900855","canonicalUrl":"https://jobsearcher.com/jobs/27d76712adf1da51d7900855","title":"Engineering Manager - Evaluation & Observability","description":"Engineering Manager – Evaluation & ObservabilityLead the team building AI evaluation and observability systems that detect drift, policy violations and prompt injection across enterprise AI.AI Governance Platform – Evaluation & ObservabilityLocationUS / RemoteExperience8-10 YearsUSD 170,000 – 190,000Posted 25 March 2026Apply by 31 July 2026About the RoleYou will build and lead the evaluation, monitoring, and observability layer of our AI governance platform — core to trust, compliance, and auditability.What You'll DoBuild systems for model evaluation (accuracy, bias, drift, hallucination detection)Build agent monitoring (inputs/outputs, decisions, tool usage)Build policy enforcement + audit logsDesign observability pipelines (real-time + batch)Define evaluation frameworks aligned with NIST AI Risk Management FrameworkWork on explainability, traceability, and audit artifactsLead a team of engineers (back-end + data + ML)What We're Looking For8–12 years engineering experience, 3+ years managing teamsFamiliarity with LLM/agent evaluation techniquesExperience building observability platformsNice to HaveExperience with AI governance / compliance systemsKnowledge of EU AI Act technical requirementsJ-18808-Ljbffr","company":"Agentsflow","rawCompany":"agentsflow","city":"Eastern","state":"KY","isRemote":false,"isActive":false,"createdAt":"2026-10-17T12:29:57.145Z","occupations":[{"code":"11-9041.00","title":"Architectural and Engineering Managers","slug":"architectural-and-engineering-managers"},{"code":"11-9199.02","title":"Compliance Managers","slug":"compliance-managers"},{"code":"11-3021.00","title":"Computer and Information Systems Managers","slug":"computer-and-information-systems-managers"}],"industries":[{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Engineering Manager - Evaluation & Observability","description":"Engineering Manager – Evaluation & ObservabilityLead the team building AI evaluation and observability systems that detect drift, policy violations and prompt injection across enterprise AI.AI Governance Platform – Evaluation & ObservabilityLocationUS / RemoteExperience8-10 YearsUSD 170,000 – 190,000Posted 25 March 2026Apply by 31 July 2026About the RoleYou will build and lead the evaluation, monitoring, and observability layer of our AI governance platform — core to trust, compliance, and auditability.What You'll DoBuild systems for model evaluation (accuracy, bias, drift, hallucination detection)Build agent monitoring (inputs/outputs, decisions, tool usage)Build policy enforcement + audit logsDesign observability pipelines (real-time + batch)Define evaluation frameworks aligned with NIST AI Risk Management FrameworkWork on explainability, traceability, and audit artifactsLead a team of engineers (back-end + data + ML)What We're Looking For8–12 years engineering experience, 3+ years managing teamsFamiliarity with LLM/agent evaluation techniquesExperience building observability platformsNice to HaveExperience with AI governance / compliance systemsKnowledge of EU AI Act technical requirementsJ-18808-Ljbffr","datePosted":"2026-10-17T12:29:57.145Z","dateModified":"2026-10-17T12:29:57.145Z","hiringOrganization":{"@type":"Organization","name":"Agentsflow","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Eastern","addressRegion":"KY","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"27d76712adf1da51d7900855"},"url":"https://jobsearcher.com/jobs/27d76712adf1da51d7900855"}}