Prompt Engineer at Checkmate New York, NY

Prompt Engineer job at Checkmate. New York, NY.In this hybrid role, you will blend traditional software engineering with natural language processing expertise to develop and maintain systems that leverage LLMs to solve real-world challenges in the restaurant tech domain. You'll collaborate with cross-functional teams to implement AI-driven features that support everything from customer service automation to order management, reporting, and staff training.Essential Job FunctionsPrompt Development & OptimizationDesign, build, test, and refine prompts for AI tasks like Q&A, classification, extraction, and synthetic data creation.Evaluation & Testing Build robust test suites using tools like CoVal, LangFuse, and Galileo to measure prompt accuracy, compliance, and performance at scale.Performance Analysis & ImprovementAnalyze prompt outputs using metrics, product feedback, and business goals. Iterate to maximize accuracy and reliability.CollaborationWork closely with operations, product, data science, and engineering teams to ensure prompts meet quality standards.Model AdaptationBy understanding their unique behaviors, create effective prompts across different models (GPT, LLaMA, Gemini, and internally fine-tuned LLMs).End-to-End OwnershipDrive prompts projects from planning to deployment, including monitoring and continuous improvement.Team Leadership & MentorshipLead a team of analysts, guiding their evaluations and data work to support prompt development. Mentor and collaborate with ops teams to ensure seamless integration of prompts into production.100% RemoteMinimum of three years of experience in the areas of AI, ML, NLP, and prompt engineering.$125,000 to $160,000RequirementsStrong Python programming skillsSolid data and computer science fundamentalsExperience with relational databases (e.g., MySQL, PostgreSQL, Oracle, MS SQL)Hands-on development experience, preferably full-stackGood data processing and testing skillsAbility to handle ambiguous problems and drive projects from start to finishStrong problem-solving mindset with a drive to improve and optimize systemsAbility to use scientific and analytical approaches to measure and evaluate designsExcellent communication skillsBachelor's degree in Computer Science, Engineering, Mathematics, or a related fieldFlexible to work US hours until at least 6 pm ET with a strong remote setupPreferred QualificationsExperience with multiple LLMs and tools like LangFuse and GalileoKnowledge of cloud platforms (AWS, GCP, Azure, OpenStack) and tools like Docker and KubernetesFocus on scalability, performance, and cost optimization in cloud environmentsBroad technical expertise across front-end and back-end systemsFamiliarity with machine learning concepts and workflowsMaster's or PhD in Computer Science, Engineering, Math, Physics, or related fieldBenefitsHealth Care Plan (Medical, Dental & Vision)Retirement Plan (401k)Life Insurance (Basic, Voluntary & AD&D)Flexible Paid Time OffFamily Leave (Maternity, Paternity)Short Term & Long Term DisabilityTraining & DevelopmentWork From HomeStock Option PlanJ-18808-Ljbffr