Engineering Manager - Evaluation & Observability
Engineering Manager – Evaluation & Observability Lead the team building AI evaluation and observability systems that detect drift, policy violations and prompt injection across enterprise AI.AI Governance Platform – Evaluation & Observability LocationUS / Remote Experience8-10 YearsUSD 170,000 – 190,000 Posted 25 March 2026 Apply by 31 July 2026
About the Role:
You will build and lead the evaluation, monitoring, and observability layer of our AI governance platform — core to trust, compliance, and auditability.
What You'll Do:
- Build systems for model evaluation (accuracy, bias, drift, hallucination detection)
- Build agent monitoring (inputs/outputs, decisions, tool usage)
- Build policy enforcement + audit logs
- Design observability pipelines (real-time + batch)
- Define evaluation frameworks aligned with NIST AI Risk Management Framework
- Work on explainability, traceability, and audit artifacts
- Lead a team of engineers (back-end + data + ML)
- What We're Looking For8–12 years engineering experience, 3+ years managing teams
- Familiarity with LLM/agent evaluation techniques
- Experience building observability platforms
Nice to Have:
Experience with AI governance / compliance systems Knowledge of EU AI Act technical requirements