Software Developer / Engineer

Job Title: Software Developer / Engineer (Mid-Level) Location: Philadelphia, PA Work Mode: Hybrid (Minimum 3 days onsite) Employment Type: Contract Interview ProcessRound 1: 1-hour In-Person InterviewRound 2: 1-hour In-Person Interview Role Overview We are seeking a mid-level Software Developer/Engineer with hands-on experience in deploying on-premise LLM solutions and vector databases. The ideal candidate will have strong expertise in Python, RAG pipelines, and enterprise-grade AI system implementation within secure environments. Key ResponsibilitiesDeploy and manage open-source LLMs (e.g., Llama 3, Mistral/Mixtral) in on-prem or private environmentsDevelop and optimize LLM inference workflows using PythonImplement Retrieval-Augmented Generation (RAG) pipelinesDesign and integrate vector database solutions for efficient semantic searchPerform model quantization and performance tuning for CPU-based inferenceEnsure data privacy, security, and governance compliance in enterprise environmentsImplement access controls, logging, and monitoring mechanismsDeliver reference architecture, prototypes, and technical documentationCollaborate with internal teams for knowledge transfer and system adoption Required Skills & QualificationsStrong experience with Python for AI/ML and backend developmentHands-on experience with open-source LLM deployment (Llama 3, Mistral, Mixtral)Experience with CPU-based inference and optimization techniquesPractical experience with vector databases (Qdrant, Chroma, Milvus, pgvector)Proven experience building RAG pipelinesKnowledge of embeddings, similarity search, and metadata filteringUnderstanding of enterprise security, data privacy, and air-gapped environments Preferred QualificationsExperience with LangChain or LlamaIndexFamiliarity with Docker and KubernetesExposure to Rust, Go, or C++ for high-performance systemsExperience with LLM inference frameworks (vLLM, llama.cpp, Hugging Face Transformers)Prior experience in regulated or enterprise environments DeliverablesEnd-to-end reference architecture for LLM + vector DB solutionsFunctional prototype (LLM + RAG + Vector DB)Comprehensive documentation and knowledge transfer