AI QA Engineer
AI QA Engineer - Fully Remote (EU or UK) - Professional Level English required
About the Company:
Our client is an innovative AI native technology company transforming the construction industry through an AI-powered cloud platform. As they continue to build an AI-native organisation, they are looking for an AI QA Engineer to help shape quality across their product.
The Role:
This is a hands-on role combining QA automation, AI/LLM testing and non-functional testing. You’ll help define testing strategy, build automation and establish the frameworks needed to test increasingly sophisticated AI and agentic features.
What you’ll do:
- Own quality strategy across functional, non-functional and AI testing.
- Build automated performance, load and stress testing using tools such as JMeter, k6 or Gatling.
- Develop and contribute to LLM/AI evaluation frameworks, assessing accuracy, hallucination, relevance, safety and other quality metrics.
- Test AI and agentic behaviours, including non-deterministic outputs, prompts, model regressions and guardrails.
- Champion shift-left and continuous testing across engineering and product.
- Establish quality metrics, observability and production quality signals.
- Use AI-assisted tools to improve testing efficiency and coverage.
- What you bring
- Strong QA foundations across test planning, functional, regression and exploratory testing.
- Practical experience or strong working knowledge of AI/LLM testing, including hallucination, consistency, safety and latency.
- Strong automation experience with Playwright, Cypress or similar, using TypeScript or Python.
- Hands-on experience with performance/load testing and CI/CD integration.
- Understanding of LLM evaluation, LLM-as-a-judge and frameworks such as DeepEval or RAGAS.Experience with API testing, microservices and cloud-native environments.
- A strong quality ownership mindset and ability to influence engineering and product teams.
- Professional level of English speaking Why Join?
- Help shape quality within an AI-native organisation.
- Have genuine influence over quality strategy and engineering practices.
- Work on AI technology transforming the construction industry.
- Build new frameworks and processes rather than simply inheriting them.
- Join a collaborative, high-growth environment with competitive compensation and flexible working.
- If you’re passionate about quality, excited by AI and want to help define the future of testing, we’d love to hear from you.