AI Workflow Analyst

We’re Hiring: AI Workflow Analyst at Sophie Society!At Sophie Society, we’re building AI tools that handle real Amazon tasks—like campaign audits, search term analysis, and listing reviews.Before anything goes live, it needs to be tested properly.This role is responsible for that. You make sure our AI outputs are accurate, useful, and safe to use on real accounts.What You’ll DoCreate test cases from real Amazon data (search term reports, campaigns, listings)Run AI tools and review the outputsCheck for: Wrong keyword classification Bad campaign structure decisions Missing or incorrect insights Hallucinated dataWrite simple scoring criteria to evaluate outputsIdentify what’s causing issues (prompt, input data, or model)Work with the builder to fix and improve resultsApprove or block tools before they go liveRequirementsStrong hands-on use of ChatGPT and ClaudeAbility to write clear prompts and improve them through testingStrong attention to detail (you catch errors others miss)Clear, structured writingComfortable with: Git, Markdown, YAML Reading basic Python or TypeScriptNice to haveExperience with Amazon PPC (campaigns, search terms, structure)Experience with tools like LangSmith, Promptfoo, or similarExperience working in an agencyWhat We OfferFully remote, comfortable work environment — work from anywhereFull-time (40h/week) — with flexibility around your schedule$35,000–$60,000 USD — based on experienceSophie Society has helped 2000+ brands, handled 2400+ different product launches and generated over $200 million in ad revenue. We are also hosts of the world's most attended digital event on Amazon PPC - The profitable PPC Challenge. Yes, loud and proud :)