Senior Data Engineer
Job Description:
Senior Engineer - Data Science10-15 YearsSan Francisco Job descriptionJob Description: Senior Engineer with Data Science + Gen AIProject FocusBuild production-grade Generative AI systems with emphasis on:RAG pipelines leveraging LLMsPrompt engineering (system prompts, tool prompts, function calling, versioning with evaluations)Evaluation & observability, including:Ground truth setupConfusion metricsHallucination rate measurementLLM-as-judge with human reviewCost & latency monitoringRetrieval strategies & prompt patternsContext managementHallucination mitigationFaithfulness & grounding validationAgentic workflowsMulti-step reasoningTool orchestrationAgent success rate measurementLangChain / LlamaIndex (or equivalent) proficiencyCloud LLM providersAzure OpenAIAWS BedrockVertex AIWorkflow orchestrationAirflowDagsterSecurity & privacyPII handlingRBACStrong Python expertise with real production experienceSuccess CriteriaProduction-ready prompt + RAG systemMeasurable quality lift (accuracy, faithfulness, confusion metrics)Reduced hallucinationsRepeatable evaluation harnessObservable system (quality, latency, cost, drift monitoring) SkillsLarge Language Models, AWS, Prompt Engineering, Python, Machine Learning, Gen AI, Confluence, Jira