Senior/Staff Software Engineer (Back-End), Inference
Overview:
In this role you will evolve the Model Vault backend to support enterprise-scale deployments with strong security and reliability. You will shape architecture across identity, deployments, billing, and observability to deliver scalable, secure model inference. You'll build and operate control-plane services that provision, configure, and manage model deployments across clouds. You will tackle complex backend and security challenges where standard SaaS patterns fall short, driving end-to-end delivery from architecture to production rollout. This is an opportunity to influence how Cohere serves large enterprises with trusted AI at scale. Compensation / Benefitsweekly lunch stipendfull health and dental benefitsRRSP matching, 401K, Pension6 weeks of paid vacationhome office stipendremote-friendly with coworking support
Responsibilities:
- Evolve and scale backend systems powering Model Vault, including authentication/authorization, deployment management, billing, metering, and customer lifecycle
- Lead major architectural initiatives across identity, payments, usage systems, platform scalability, and other core product foundations
- Build highly reliable control-plane services for provisioning, configuring, scaling, and operating model deployments across clouds and hardware platforms
- Design APIs and distributed systems with guarantees around correctness, consistency, isolation, failure recovery, idempotency, and observability
- Incorporate security and privacy into platform architecture (tenant isolation, least-privilege, auditability, secrets management, encryption, confidential computing)
- Solve complex problems spanning backend services, cloud infrastructure, security, and AI inference beyond conventional SaaS patterns
- Drive projects end-to-end from architecture and threat modeling through implementation, rollout, production operations, and iteration based on customer workloads
Key requirements:
- Strong backend engineer with fundamentals in distributed systems, databases, networking, operating systems, and security
- Proficiency in production systems using Go, Python, Java, C++, Rust, or similar languages
- Experience designing and operating production systems with emphasis on correctness, availability, security, and performance at scale
- Ability to define architecture for ambiguous, high-impact problems and deliver reliable production systems
- Solid systems-design instincts: concurrency, consistency, caching, queues, rate limiting, backpressure, failure domains, idempotency, graceful degradation
- Awareness of security boundaries: authentication, authorization, identity, cryptography, secrets, tenant isolation, security risks
- Comfort across backend services, APIs, databases, distributed systems, and cloud infrastructure
- Willingness to dive into unfamiliar codebases and debug cross-boundary issues
- Attention to production engineering details: instrumentation, capacity planning, migrations, backward compatibility, incident response, safe rollouts
- Track record delivering technically difficult projects that become reliable for engineers and customersownership mindsetability to drive cross-functional initiativesstrong analytical and problem-solving abilities
GoPythonJava