{"schemaVersion":"jobsearcher.job.v1","id":"32d9949090a57fde8f8b8b95","url":"https://jobsearcher.com/jobs/32d9949090a57fde8f8b8b95","canonicalUrl":"https://jobsearcher.com/jobs/32d9949090a57fde8f8b8b95","title":"AI Implementation Quality Analyst","description":"The Company:\nServing the People Who Serve the People\n\nGranicus is driven by the excitement of building, implementing, and maintaining technology that is transforming the Govtech industry by bringing governments and its constituents together. We are on a mission to support our customers with meeting the needs of their communities and implementing our technology in ways that are equitable and inclusive. Granicus has consistently appeared on the GovTech 100 list over the past 5 years and has been recognized as the best companies to work on BuiltIn.\n\nOver the last 25 years, we have served 5,500 federal, state, and local government agencies and more than 300 million citizen subscribers power an unmatched Subscriber Network that use our digital solutions to make the world a better place. With comprehensive cloud-based solutions for communications, government website design, meeting and agenda management software, records management, and digital services, Granicus empowers stronger relationships between government and residents across the U.S., U.K., Australia, New Zealand, and Canada. By simplifying interactions with residents, while disseminating critical information, Granicus brings governments closer to the people they serve—driving meaningful change for communities around the globe.\n\nWant to know more? See more of what we do here.\nJob Summary:\nThe AI Quality & Evaluation Analyst is responsible for assessing the quality, correctness, completeness, and safety of AI‑generated responses across defined use cases. This role combines hands‑on human review with structured, rubric‑based evaluation incorporating automation to ensure AI systems meet documented standards before customer implementations go live.\nAs part of the implementation team, this role is client-facing and serves as a bridge between client expectations and system behavior, translating real-world use cases, domain context, and risk tolerance into measurable evaluation criteria and actionable feedback for product and engineering teams.\nThe role focuses on what the AI says and does, not on model training or infrastructure performance. The analyst serves as a human quality gate, ensuring outputs are accurate, appropriate for the audience, and aligned with policy and governance requirements.\nThis role works closely with product and domain experts to translate real‑world expectations into measurable evaluation criteria and repeatable test artifacts.\n\nWhat Your Impact Will Look Like:\n\nPartner directly with clients during implementation to understand use cases, success criteria, and risk tolerance\nTranslate client requirements into evaluation frameworks, prompt strategies, and test coverage\nAct as the quality liaison between client, product, and engineering to ensure alignment pre- and post-launch\nReview and score AI responses using standard rubrics\nValidate against sources/ground truth; flag hallucinations, omissions, and other risks\nDocument findings and calibrate with peers to keep scoring consistent\nBuild and maintain prompt banks and golden sets (expected results)\nExpand coverage for edge cases, high-risk scenarios, and real user language\nTrack regression/drift and feed dashboards and quality reports\nTriage internal and client feedback; synthesize themes across deployments\nIdentify systemic risks and escalate high-impact findings with clear, client-relevant context\nPartner with product/engineering to validate and verify fixes, ensuring alignment with client expectations\nYou Will Love This Job If You Have:\nStrong analytical judgment and attention to detail\nExcellent written communication skills, including the ability to explain reasoning clearly\nExperience reviewing, auditing, or evaluating structured outputs such as content, decisions, or recommendations\nComfort applying detailed guidelines and rubrics consistently at scale\nFamiliarity with large language models and common failure modes such as hallucinations, overgeneralization, or unsafe responses\nAbility to work in spreadsheets, evaluation tools, or annotation platforms\nExperience with AI evaluation, data annotation, QA, trust and safety, or policy review\nExposure to human‑in‑the‑loop workflows, or benchmarking processes\nDomain expertise in a regulated or high‑risk field such as government, education, healthcare, or legal services\nExperience contributing to test suites, evaluation dashboards, or quality reporting\nComfort working cross‑functionally with product and engineering teams\nExperience in client-facing roles such as implementation, consulting, or solution delivery in a SaaS or regulated environment\nAbility to translate between business/user needs and technical system behavior\n\nAI responses consistently meet defined standards for accuracy, completeness, and safety in real client use cases\nEvaluation results are reproducible and trusted across teams and with clients\nClient requirements and risk tolerances are clearly translated into evaluation criteria and test coverage\n\nQuality issues are identified early, clearly documented, and actionable\nTest coverage expands over time to reflect real user behavior and risk\nStakeholders can confidently use evaluation outputs to make release and governance decisions\nAI outputs are consistently accurate, complete, and safe in real client use cases, with quality standards that are clearly defined and trusted across teams. Client expectations and risk tolerances are effectively translated into repeatable evaluation frameworks and test coverage. Issues are identified early, communicated clearly, and resolved quickly with engineering. Evaluation outputs are reliable and actionable, enabling confident release decisions and building trust with both internal stakeholders and clients.\n\nA strong instinct for what “good” looks like and the discipline to define and measure it. You enjoy digging into ambiguous outputs, spotting subtle issues, and backing your judgment with clear reasoning and evidence. You’re comfortable operating at the intersection of clients and technical teams—translating real-world needs into structured evaluation criteria and turning findings into actionable improvements. You like building systems that scale (rubrics, test sets, coverage) and take pride in making complex behavior consistent, reliable, and trustworthy.\nPay Range: USD $80,000.00 - USD $105,700.00 /Yr. About Us:\nDon’t have all the skills/experience mentioned above? At Granicus, we are trying to build diverse, inclusive teams. We do not have degree requirements for most of our roles. If you don’t meet every requirement above but are excited to learn more, we encourage you to apply. We might just be able to find another role that could be a perfect fit!\n\nSecurity and Privacy Requirements\nResponsible for Granicus information security by appropriately preserving the Confidentiality, Integrity, and Availability (CIA) of Granicus information assets in accordance with the company's information security program.\nResponsible for ensuring the data privacy of our employees and customers, their data, as well as taking all required privacy training in a timely manner, in accordance with company policies.\n\nThe Team\nWe are a remote-first company with a globally distributed workforce across the United States, Canada, United Kingdom, India, Armenia, Australia, and New Zealand.\n\nThe Culture\nAt Granicus, we are building a transparent, inclusive, and safe space for everyone who wants to be\na part of our journey.\nA few culture highlights include – Employee Resource Groups to encourage diverse voices\nCoffee with Mark sessions – Our employees get to interact with our CEO on very important and\nsometimes difficult issues ranging from mental health to work-life balance and current affairs.\nMicrosoft Teams communities focused on wellness, art, furbabies, family, parenting, and more.\nWe bring in special guests from time to time to discuss issues that impact our employee\npopulation\n\nThe Impact\nWe are proud to serve dynamic organizations around the globe that use our digital solutions to make the world a better place — quite literally. We have so many powerful success stories that illustrate how our solutions are impacting the world. See more of our impact here.\nThe Benefits: At Granicus, we offer a comprehensive and flexible benefits package designed to support your well-being, growth, and work-life balance—starting from day one.\n\nHere’s what you can expect as a U.S.-based team member:\nFlexibility & Balance\nFlexible Time Off – Take the time you need to rest, recharge, and live your life.\nCompany-Wide Wellbeing Days – Paid days off to unplug and focus on your mental health.\nWork From Home Reimbursement – Support a productive home office environment.\nHealth & Wellness\nMultiple Health Plan Options – Including a 100% employer-paid plan.\nEmployer HSA Contributions – When enrolled in a High-Deductible Health Plan.\nFitness Reimbursement Program – Stay active, your way.\nOn-Demand Mental Health Support – Access to Headspace and other wellness tools.\nFamily & Future\nPaid Parental Leave – For both birthing and non-birthing parents.\nTraditional & Roth 401(k) – With a generous company match.\nLife & AD&D Insurance – 100% employer-paid coverage for peace of mind.\nGrowth & Recognition\nOnline Learning Platforms – Fuel your professional development.\nCompetitive Salary & Bonuses – Your contributions are valued and rewarded.\nEqual Opportunity Employer: Granicus is committed to providing equal employment opportunities. All qualified applicants and employees will be considered for employment and advancement without regard to race, color, religion, creed, national origin, ancestry, sex, gender, gender identity, gender expression, physical or mental disability, age, genetic information, sexual or affectional orientation, marital status, status with regard to public assistance, familial status, military or veteran status or any other status protected by applicable law.","company":"Granicus","rawCompany":"granicus","city":"Remote","state":"OR","isRemote":false,"isActive":false,"createdAt":"2026-08-05T12:43:07.517Z","occupations":[{"code":"15-1253.00","title":"Software Quality Assurance Analysts and Testers","slug":"software-quality-assurance-analysts-and-testers"},{"code":"19-4099.01","title":"Quality Control Analysts","slug":"quality-control-analysts"},{"code":"11-3051.01","title":"Quality Control Systems Managers","slug":"quality-control-systems-managers"}],"industries":[{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541690","title":"Other Scientific and Technical Consulting Services","slug":"other-scientific-and-technical-consulting-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"AI Implementation Quality Analyst","description":"The Company:\nServing the People Who Serve the People\n\nGranicus is driven by the excitement of building, implementing, and maintaining technology that is transforming the Govtech industry by bringing governments and its constituents together. We are on a mission to support our customers with meeting the needs of their communities and implementing our technology in ways that are equitable and inclusive. Granicus has consistently appeared on the GovTech 100 list over the past 5 years and has been recognized as the best companies to work on BuiltIn.\n\nOver the last 25 years, we have served 5,500 federal, state, and local government agencies and more than 300 million citizen subscribers power an unmatched Subscriber Network that use our digital solutions to make the world a better place. With comprehensive cloud-based solutions for communications, government website design, meeting and agenda management software, records management, and digital services, Granicus empowers stronger relationships between government and residents across the U.S., U.K., Australia, New Zealand, and Canada. By simplifying interactions with residents, while disseminating critical information, Granicus brings governments closer to the people they serve—driving meaningful change for communities around the globe.\n\nWant to know more? See more of what we do here.\nJob Summary:\nThe AI Quality & Evaluation Analyst is responsible for assessing the quality, correctness, completeness, and safety of AI‑generated responses across defined use cases. This role combines hands‑on human review with structured, rubric‑based evaluation incorporating automation to ensure AI systems meet documented standards before customer implementations go live.\nAs part of the implementation team, this role is client-facing and serves as a bridge between client expectations and system behavior, translating real-world use cases, domain context, and risk tolerance into measurable evaluation criteria and actionable feedback for product and engineering teams.\nThe role focuses on what the AI says and does, not on model training or infrastructure performance. The analyst serves as a human quality gate, ensuring outputs are accurate, appropriate for the audience, and aligned with policy and governance requirements.\nThis role works closely with product and domain experts to translate real‑world expectations into measurable evaluation criteria and repeatable test artifacts.\n\nWhat Your Impact Will Look Like:\n\nPartner directly with clients during implementation to understand use cases, success criteria, and risk tolerance\nTranslate client requirements into evaluation frameworks, prompt strategies, and test coverage\nAct as the quality liaison between client, product, and engineering to ensure alignment pre- and post-launch\nReview and score AI responses using standard rubrics\nValidate against sources/ground truth; flag hallucinations, omissions, and other risks\nDocument findings and calibrate with peers to keep scoring consistent\nBuild and maintain prompt banks and golden sets (expected results)\nExpand coverage for edge cases, high-risk scenarios, and real user language\nTrack regression/drift and feed dashboards and quality reports\nTriage internal and client feedback; synthesize themes across deployments\nIdentify systemic risks and escalate high-impact findings with clear, client-relevant context\nPartner with product/engineering to validate and verify fixes, ensuring alignment with client expectations\nYou Will Love This Job If You Have:\nStrong analytical judgment and attention to detail\nExcellent written communication skills, including the ability to explain reasoning clearly\nExperience reviewing, auditing, or evaluating structured outputs such as content, decisions, or recommendations\nComfort applying detailed guidelines and rubrics consistently at scale\nFamiliarity with large language models and common failure modes such as hallucinations, overgeneralization, or unsafe responses\nAbility to work in spreadsheets, evaluation tools, or annotation platforms\nExperience with AI evaluation, data annotation, QA, trust and safety, or policy review\nExposure to human‑in‑the‑loop workflows, or benchmarking processes\nDomain expertise in a regulated or high‑risk field such as government, education, healthcare, or legal services\nExperience contributing to test suites, evaluation dashboards, or quality reporting\nComfort working cross‑functionally with product and engineering teams\nExperience in client-facing roles such as implementation, consulting, or solution delivery in a SaaS or regulated environment\nAbility to translate between business/user needs and technical system behavior\n\nAI responses consistently meet defined standards for accuracy, completeness, and safety in real client use cases\nEvaluation results are reproducible and trusted across teams and with clients\nClient requirements and risk tolerances are clearly translated into evaluation criteria and test coverage\n\nQuality issues are identified early, clearly documented, and actionable\nTest coverage expands over time to reflect real user behavior and risk\nStakeholders can confidently use evaluation outputs to make release and governance decisions\nAI outputs are consistently accurate, complete, and safe in real client use cases, with quality standards that are clearly defined and trusted across teams. Client expectations and risk tolerances are effectively translated into repeatable evaluation frameworks and test coverage. Issues are identified early, communicated clearly, and resolved quickly with engineering. Evaluation outputs are reliable and actionable, enabling confident release decisions and building trust with both internal stakeholders and clients.\n\nA strong instinct for what “good” looks like and the discipline to define and measure it. You enjoy digging into ambiguous outputs, spotting subtle issues, and backing your judgment with clear reasoning and evidence. You’re comfortable operating at the intersection of clients and technical teams—translating real-world needs into structured evaluation criteria and turning findings into actionable improvements. You like building systems that scale (rubrics, test sets, coverage) and take pride in making complex behavior consistent, reliable, and trustworthy.\nPay Range: USD $80,000.00 - USD $105,700.00 /Yr. About Us:\nDon’t have all the skills/experience mentioned above? At Granicus, we are trying to build diverse, inclusive teams. We do not have degree requirements for most of our roles. If you don’t meet every requirement above but are excited to learn more, we encourage you to apply. We might just be able to find another role that could be a perfect fit!\n\nSecurity and Privacy Requirements\nResponsible for Granicus information security by appropriately preserving the Confidentiality, Integrity, and Availability (CIA) of Granicus information assets in accordance with the company's information security program.\nResponsible for ensuring the data privacy of our employees and customers, their data, as well as taking all required privacy training in a timely manner, in accordance with company policies.\n\nThe Team\nWe are a remote-first company with a globally distributed workforce across the United States, Canada, United Kingdom, India, Armenia, Australia, and New Zealand.\n\nThe Culture\nAt Granicus, we are building a transparent, inclusive, and safe space for everyone who wants to be\na part of our journey.\nA few culture highlights include – Employee Resource Groups to encourage diverse voices\nCoffee with Mark sessions – Our employees get to interact with our CEO on very important and\nsometimes difficult issues ranging from mental health to work-life balance and current affairs.\nMicrosoft Teams communities focused on wellness, art, furbabies, family, parenting, and more.\nWe bring in special guests from time to time to discuss issues that impact our employee\npopulation\n\nThe Impact\nWe are proud to serve dynamic organizations around the globe that use our digital solutions to make the world a better place — quite literally. We have so many powerful success stories that illustrate how our solutions are impacting the world. See more of our impact here.\nThe Benefits: At Granicus, we offer a comprehensive and flexible benefits package designed to support your well-being, growth, and work-life balance—starting from day one.\n\nHere’s what you can expect as a U.S.-based team member:\nFlexibility & Balance\nFlexible Time Off – Take the time you need to rest, recharge, and live your life.\nCompany-Wide Wellbeing Days – Paid days off to unplug and focus on your mental health.\nWork From Home Reimbursement – Support a productive home office environment.\nHealth & Wellness\nMultiple Health Plan Options – Including a 100% employer-paid plan.\nEmployer HSA Contributions – When enrolled in a High-Deductible Health Plan.\nFitness Reimbursement Program – Stay active, your way.\nOn-Demand Mental Health Support – Access to Headspace and other wellness tools.\nFamily & Future\nPaid Parental Leave – For both birthing and non-birthing parents.\nTraditional & Roth 401(k) – With a generous company match.\nLife & AD&D Insurance – 100% employer-paid coverage for peace of mind.\nGrowth & Recognition\nOnline Learning Platforms – Fuel your professional development.\nCompetitive Salary & Bonuses – Your contributions are valued and rewarded.\nEqual Opportunity Employer: Granicus is committed to providing equal employment opportunities. All qualified applicants and employees will be considered for employment and advancement without regard to race, color, religion, creed, national origin, ancestry, sex, gender, gender identity, gender expression, physical or mental disability, age, genetic information, sexual or affectional orientation, marital status, status with regard to public assistance, familial status, military or veteran status or any other status protected by applicable law.","datePosted":"2026-08-05T12:43:07.517Z","dateModified":"2026-08-05T12:43:07.517Z","hiringOrganization":{"@type":"Organization","name":"Granicus","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Remote","addressRegion":"OR","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"32d9949090a57fde8f8b8b95"},"url":"https://jobsearcher.com/jobs/32d9949090a57fde8f8b8b95"}}