JOBSEARCHER

Software Engineer - Hosted Model Infrastructure

Overview: In this role you contribute to building high-performance ML model serving infrastructure and deployment pipelines, enabling secure, scalable AI across on-prem, air-gapped, and constrained environments. You work across the stack—from inference engines to orchestration and observability—delivering rapid, reliable model rollouts for critical customers. You collaborate with product teams to translate customer needs into robust deployments and continuous improvements. This is an impactful, hands-on position at Palantir where infrastructure meets cutting-edge AI in demanding contexts. Compensation / Benefitsmedical, dental, and vision insurance 401k planrelocation assistance2 weeks paid time off annually10 paid holidayspaid parental leave
Responsibilities: Build high-performance model serving infra aligned with security models and hardware constraintsDesign intelligent request handling for multi-tenant access (auth, rate limiting, concurrency, auditing) Create and maintain packaging/deployment pipelines for on-prem and air-gapped environmentsDevelop production observability for AI systems to enable monitoring and incident triageDebug complex issues across the stack including inference engines, containers, and GPU driversDesign and run testing/benchmarking infra to validate deployments across GPU hardwareCollaborate with product teams to capture requirements, debug production issues, deliver capabilitiesIntegrate hosted model infra with Palantir deployment, configuration, and identity systems
Key requirements: 4+ years of professional software engineering experience building/operating production systemsEngineering background in CS, Math, Software Eng, Physics or similarProficiency in Java, C++, Python, Rust, or similar; familiarity with Python ML ecosystemExperience with containers, Kubernetes, and production backend deploymentStrong communication skills and ability to iterate quickly with teammates and maintain high qualityOwnership mindsetcustomer empathycross-language collaborationJavaRustPython