{"schemaVersion":"jobsearcher.job.v1","id":"01f2f743e484664a5506b41e","url":"https://jobsearcher.com/jobs/01f2f743e484664a5506b41e","canonicalUrl":"https://jobsearcher.com/jobs/01f2f743e484664a5506b41e","title":"AI DevOps / Observability Engineer","description":"We are seeking an experienced AI DevOps/Observability Engineer to support production operations for Generative AI and LLM agent applications. The role focuses on reliability, observability, automated evaluation, performance optimization, and production readiness.Key ResponsibilitiesImplement LLM tracing, logging, telemetry, and observability for AI/agent workflows.Build production dashboards for infrastructure, application, and AI performance metrics.Define SLIs/SLOs, actionable alerts, runbooks, and production-readiness standards.Develop automated LLM evaluation/testing pipelines for quality, safety, hallucination, faithfulness, and relevance.Analyze prompts, token usage, latency, model performance, cost, and accuracy.Support CI/CD, deployments, incident management, and production reliability.Required SkillsLLM/Agent Evaluation: Ragas, DeepEval, TruLens or similar.LLM Observability/Tracing: LangSmith, Langfuse, Phoenix, Arize, OpenTelemetry.Monitoring: Datadog, Prometheus/Grafana, New Relic, or cloud-native monitoring.SRE/Operations: SLI/SLO definition, alerting, incident management, runbooks.Test Automation: Automated testing integrated with CI/CD pipelines.AI/Model Analysis: Prompt benchmarking, regression testing, latency and cost optimization.Python: Advanced Python, async programming, APIs, LangChain/LlamaIndex.Cloud & DevOps: AWS/GCP/Azure, Docker, Kubernetes, CI/CD.Preferred Qualifications3+ years operationalizing LLM/Generative AI applications in production.Experience with Vector Databases such as Pinecone, Milvus, or Chroma.Experience monitoring and optimizing RAG pipelines.Background in SRE, MLOps, or AI production operations.","company":"Arkhya Tech","rawCompany":"arkhya tech","city":"Long Beach Twp","state":"NJ","isRemote":false,"isActive":false,"createdAt":"2026-10-01T22:55:07.999Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1244.00","title":"Network and Computer Systems Administrators","slug":"network-and-computer-systems-administrators"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"AI DevOps / Observability Engineer","description":"We are seeking an experienced AI DevOps/Observability Engineer to support production operations for Generative AI and LLM agent applications. The role focuses on reliability, observability, automated evaluation, performance optimization, and production readiness.Key ResponsibilitiesImplement LLM tracing, logging, telemetry, and observability for AI/agent workflows.Build production dashboards for infrastructure, application, and AI performance metrics.Define SLIs/SLOs, actionable alerts, runbooks, and production-readiness standards.Develop automated LLM evaluation/testing pipelines for quality, safety, hallucination, faithfulness, and relevance.Analyze prompts, token usage, latency, model performance, cost, and accuracy.Support CI/CD, deployments, incident management, and production reliability.Required SkillsLLM/Agent Evaluation: Ragas, DeepEval, TruLens or similar.LLM Observability/Tracing: LangSmith, Langfuse, Phoenix, Arize, OpenTelemetry.Monitoring: Datadog, Prometheus/Grafana, New Relic, or cloud-native monitoring.SRE/Operations: SLI/SLO definition, alerting, incident management, runbooks.Test Automation: Automated testing integrated with CI/CD pipelines.AI/Model Analysis: Prompt benchmarking, regression testing, latency and cost optimization.Python: Advanced Python, async programming, APIs, LangChain/LlamaIndex.Cloud & DevOps: AWS/GCP/Azure, Docker, Kubernetes, CI/CD.Preferred Qualifications3+ years operationalizing LLM/Generative AI applications in production.Experience with Vector Databases such as Pinecone, Milvus, or Chroma.Experience monitoring and optimizing RAG pipelines.Background in SRE, MLOps, or AI production operations.","datePosted":"2026-10-01T22:55:07.999Z","dateModified":"2026-10-01T22:55:07.999Z","hiringOrganization":{"@type":"Organization","name":"Arkhya Tech","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Long Beach Twp","addressRegion":"NJ","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"01f2f743e484664a5506b41e"},"url":"https://jobsearcher.com/jobs/01f2f743e484664a5506b41e"}}