AI 101 Capstone Project Reviewer & Learning Evaluator

Brief DescriptionLaunchCode is hiring a part-time contract reviewer to evaluate student capstone projects for our AI 101 course bringing the expert judgment, real-world experience, and developmental feedback that AI alone can't provide.At a GlanceType: Part-Time 1099 Contractor — Remote (all U.S. states except CA)Compensation: $30/submission flat rateVolume: ~250 submissions estimated, 20–30 min each (est. 80–125 hours total)Experience: 3–7 years hands-on automation experience, or equivalentSchedule: Flexible; work on your own time within review windows_*]:min-w-0 gap-3 !gap-3.5">About This OpportunityLaunchCode's AI 101 course teaches professionals to build AI-powered automations. The course culminates in a capstone project where students build a Zapier-based automation, make a business case for its value, and present their work as an 8–12 slide PDF deck. An AI agent scores each submission across ten rubric criteria spanning two categories (Content and Presentation), then generates written feedback. Your job is to validate that output—correcting scores, rewriting weak feedback, and applying the professional judgment the AI can't. Students apply four analytical frameworks across their projects; onboarding covers each in detail. The central challenge of this role is anchoring bias: reviewers who lack domain depth tend to accept the AI's score rather than override it, even when it's wrong. We need someone with enough expertise across automation design, business case construction, measurement methodology, and responsible AI to catch what the AI gets wrong—especially at the Proficient/Exemplary boundary, where the difference requires real professional judgment.Key ResponsibilitiesValidate AI scores across all ten rubric criteria; adjust where the AI has mis-scored or missed nuanceRewrite AI feedback that is generic, unclear, or misses a learning opportunity — feedback must be specific and actionableApply particular scrutiny to: comparative analysis scoring (AI over-credits surface-level comparisons); measurement data quality (AI accepts weak methodology); and responsible AI scoring (AI awards credit for naming risks without a credible mitigation plan)Distinguish Picking the Right Problem (task selection quality) from Strategic Automation (step prioritization quality) — these are separate rubric criteria and must be scored independentlyFlag submissions with structural gaps the AI may miss: missing architecture slide, absent speaker notes, slide count outside 8–12, or exposed credentials (API keys, tokens) in screenshotsFlag systemic AI scoring patterns to the course team; participate in periodic calibration checksRequired Skills & QualificationsZapier (required): Must be able to evaluate multi-step Zaps and Agents across all five workflow components — trigger, actions, handoffs, data mapping, and output — and distinguish a functional workflow (Proficient) from one that handles edge cases and could be handed to a colleague with minimal setup (Exemplary)Workflow & automation design: Able to assess architecture diagrams, identify logical gaps, and evaluate whether automation choices target the highest-impact stepsBusiness acumen: Able to evaluate ROI cases, including the distinction between capacity savings and direct cost savings; comfortable assessing whether a student's projected annual value is credibleData literacy: Able to assess whether deployment data (quality, speed, cost) is credible, methodology is sound, and data visualizations actively support the argument rather than just accompany itResponsible AI awareness: Familiar with data privacy, algorithmic bias, human-in-the-loop design, and graceful failure documentation; able to distinguish a genuine mitigation plan from a performative oneAI output evaluation: Experience QA'ing AI-generated scores or assessments in a professional context is required; you must be comfortable overriding AI judgment and articulating whyPresentation evaluation: Able to assess narrative arc, visual design quality, and slide-and-speaker-note coherence for a non-technical business audienceWritten communication: Able to write developmental feedback for adult learners that is specific, encouraging, and growth-orientedPreferred Skills & QualificationsExperience teaching, coaching, or providing feedback in a bootcamp, corporate training, or adult learning contextBackground in operations, consulting, or a field where you've personally built and defended automation ROI casesFamiliarity with other automation tools (Make, n8n, Power Automate) in addition to ZapierExperience evaluating presentations or slide decks in a professional review capacity