Senior QA Tester – GenAI/LLM Testing

Job Title: Senior QA Tester – GenAI/LLM TestingLocation: Houston, TX (Onsite/Hybrid)Experience: 7+ YearsJob Description:We are seeking an experienced Senior QA Tester with deep expertise in testing Large Language Model (LLM)-based applications, GenAI, and ML-driven products. The ideal candidate should have strong hands-on experience with LangSmith, Promptfoo, and other modern LLM testing frameworks, along with a proven track record in automation, functional, and performance testing.Key Responsibilities:   •   Design, develop, and execute test strategies for LLM-based products, Generative AI applications, and ML workflows.   •   Perform prompt testing, regression testing, functional testing, and evaluation of LLM-powered features using tools like LangSmith and Promptfoo.   •   Validate model outputs, hallucinations, bias, accuracy, and reliability of AI responses.   •   Develop automated test scripts for GenAI/ML pipelines to ensure scalability and consistency.   •   Work closely with developers, ML engineers, and product teams to identify defects, edge cases, and quality gaps in AI-driven features.   •   Collaborate on CI/CD integration for automated testing of AI/ML models and APIs.   •   Document and maintain test cases, bug reports, and test execution results.   •   Provide technical expertise and best practices for AI/ML model validation and benchmarking.Required Skills & Experience:   •   7+ years of hands-on experience in QA Testing, with at least 2+ years in GenAI/LLM/ML product testing.   •   Proficiency in LLM testing frameworks such as LangSmith and Promptfoo.   •   Strong knowledge of Generative AI models, NLP, and ML pipelines.   •   Solid expertise in test automation (Python, Pytest, Java, or similar frameworks).   •   Experience testing APIs, microservices, and cloud-based AI solutions (AWS, Azure, GCP).   •   Understanding of prompt engineering, evaluation metrics, and model fine-tuning validation.   •   Strong debugging, problem-solving, and analytical skills.   •   Excellent communication skills and ability to work in a fast-paced, agile environment.Nice-to-Have:   •   Experience with Apple ecosystem tools (Xcode, XCTest, iOS/macOS testing).   •   Familiarity with MLOps practices and model monitoring frameworks.   •   Exposure to RAG (Retrieval Augmented Generation) or vector database testing.