JOBSEARCHER

QA Analyst (AI Systems)

ARCHIVED

We can't find an active application page for this role right now. It may reopen or be listed elsewhere. Use Next Steps to search for an active apply link and similar live jobs.

About CodeRoadCodeRoad provides end-to-end software development services, helping businesses scale with ideal infrastructure solutions. From staff augmentation to dedicated IT teams and general software engineering, our nearshore technology services empower businesses to thrive in an ever-evolving digital landscape.
About the Role: We’re looking for a QA Analyst specialized in modern AI systems. You’ll be responsible for validating not only APIs and performance, but also the behavior and reliability of AI-driven outputs. This role goes beyond traditional QA — you’ll test intelligent systems, not just deterministic ones.
Key Responsibilities:
  • Design and execute test strategies for AI-powered APIs and workflows
  • Validate agent inputs/outputs and ensure consistency and reliability
  • Perform API testing, exploratory testing, and automated testing
  • Run performance, load, and bulk testing scenarios
  • Collaborate with engineers to identify edge cases in AI behavior
  • Contribute to testing frameworks for LLM-based systems
Requirements: 3+ years of experience in QA (Automation + Manual).Strong API Testing: Expert level with Postman, Insomnia, or Rest Assured. Automation Specialist: Proficient in Playwright or Selenium for end-to-end flows. Performance & Scale: Experience with Load/Bulk testing using Locust or JMeter. AI/LLM Validation: Familiarity with testing Agentic workflows, validating AI payloads, and using tools like LangSmith. Observability: Exposure to Open Telemetry for tracing and system monitoring. Analytical Rigor: Ability to design test cases for non-linear, agentic AI tasks. Advanced English: Fluent for documentation and global collaboration.
Nice to Have:
  • Experience testing AI/ML systems
  • Familiarity with LangSmith or similar tools
  • Experience with MLFlow or model evaluation frameworks
  • What You’ll Love 100% Remote
  • Holidays off
  • Paid Time OffHealth insurance assistance program
  • Competitive USD compensation
  • Strong team culture and collaborative environment
  • Ongoing training and growth opportunities