JOBSEARCHER

Game Engineer

Own the AI safety layer for Realtor.com's RealAssist, an LLM-powered real-estate assistant used across web and mobile. On the AI Integrations Team at News Corp, you will build and tune LLM safety guardrails that act as a launch gate for Fair Housing compliance and content moderation, with runtime prompt-injection and jailbreak screening designed for production.
What you'll do:
  • Build, tune, and take end-to-end ownership of LLM-as-a-judge classifiers for Fair-Housing compliance and content moderation, targeting high recall on disallowed content without overblocking legitimate users
  • Design and run guardrail evaluation pipelines, including labeled dataset curation, train/test/validate splits, and offline prod-replay to evaluate overblocking
  • Use defensible metrics by reporting confusion matrices and precision/recall/F1 per category to guide safety decisions
  • Integrate and operate cloud guardrail services as a runtime prompt-injection/jailbreak screening layer, including fail-open behavior and alerting
  • Red-team the assistant and wire safety regression checks into CI to prevent silent guardrail degradation as the product grows
  • Manage guardrail infrastructure as code using Terraform (templates, IAM, and project shape), and participate in technical design reviews and architecture discussions
  • Partner with ML, backend, product, and legal/compliance stakeholders to define risk tiering and get new AI capabilities through safety review before launch
  • Independently design, implement, and tune guardrail and evaluation systems with a focus on balancing safety, latency, and user experience; write clean, maintainable, well-tested code
  • Keep eval datasets calibrated to real traffic and emerging attack patterns, staying current with AI-safety research and new guardrail tooling
  • What you bringAI Safety & Evaluation Expertise (required)4+ years of professional software/ML engineering experience with hands-on LLM application work
  • Bachelor's degree or equivalent experience
  • Strong Python proficiency
  • Real classification-metrics literacy: precision/recall/F1 (macro vs weighted), confusion matrix debugging, dataset curation, and calibration to production distribution
  • Experience evaluating LLM prompts/classifiers with an eval framework such as DeepEval, RAGAS, Arize Phoenix, LangSmith, OpenAI Evals, or a homegrown harness
  • Ability to turn small seed sets into robust labeled eval datasets using dataset-synthesis and prompt-optimization loops
  • Familiarity with prompt-injection/jailbreak defense concepts, including OWASP LLM Top 10, input/output filtering, least-privilege tool access, and adversarial testing
  • Hands-on experience with cloud guardrail services: Google Cloud Model ArmorAWS Bedrock Guardrails, or Azure AI Content Safety
  • Terraform/IaC for cloud infrastructure
  • Experience with Google Cloud Vertex AI / Gemini
  • Understanding of monitoring and observability tools (for example, New Relic or similar)
  • Exposure to regulated, compliance-sensitive domains (fair housing, fair lending, healthcare, finance, trust & safety)
  • Red-teaming or AI-security background
  • Tools you'll work with
  • PythonDeepEvalRAGASArize Phoenix
  • LangSmithOpenAI EvalsOWASP LLM Top 10 Google Cloud Model ArmorAWS Bedrock Guardrails
  • Azure AI Content Safety
  • TerraformGoogle Cloud Vertex AI / GeminiNew RelicOnsite role with strong support and collaboration
  • The AI Integrations Team is a cross-functional squad building how Realtor.com integrates AI across its products. The work blends creativity and innovation with in-person collaboration. For most roles, employees work four or more days in office
Benefits:
  • Inclusive and competitive medical, Rx, dental, and vision coverage
  • Family forming benefits13 Paid Holidays
  • Flexible Time Off8 hours of paid volunteer time off
  • Immediate eligibility into the Company 401(k) plan with 3.5% company match
  • Tuition Reimbursement for degreed and non-degreed programs1:1 personalized Financial Planning Sessions
  • Free snacks and refreshments in each office location
  • Student Debt Retirement Savings Match program