Sr. Principal Software Engineer

A Moving Experience.Who is Cerence AI?Cerence AI is the global leader in AI for transportation, specialized in building AI and voice-powered companions for cars, two-wheelers, and more that enable people to focus on what matters most. With over 500 million cars shipped with Cerence AI's technology, we partner with leading automakers (such as Volkswagen, Mercedes, Audi, Toyota and many more), mobility providers, and technology companies to power intuitive, integrated experiences that create safer, more connected, and more enjoyable journeys for drivers and passengers alike.Our Driving ForceOur team is dedicated to pushing the boundaries of AI innovation, working around the globe with headquarters in Burlington, Massachusetts, USA and 16 other offices across Europe, Asia, and North America. We bring together diverse backgrounds, and varied skill sets with the shared goal of advancing the next generation of transportation user experiences. Our culture is customer-centric, collaborative, fast-paced, and fun, with continuous opportunities for learning and development to support your career growth.Interested in having a significant impact in a dynamic industry with a high-performing global team? We're looking for an exceptional Senior Principal Software Engineer who is ready to drive the future of mobility with us!Job Description:What You Will Work OnOptimize and deploy high-performance LLM inference pipelinesOwn inference runtimes across data center, edge, and embedded platformsPush model performance through quantization, kernel fusion, and cache optimizationDrive latency and throughput improvements that directly impact production productsEnable efficient, reliable deployment without external vendor dependencyCore ResponsibilitiesInference Engines & RuntimeBuild deep expertise and ownership of:vLLMTensorRT-LLMllama.cppQAIRTExtend and tune inference engines using custom CUDA kernelsAdapt runtimes for constrained and embedded deployment environmentsQuantization & Numerical OptimisationImplement and evaluate quantisation strategies:INT8, INT4, FP4, FP8, mixed precisionAWQGPTQBalance accuracy, latency, memory footprint, and throughputKV Cache OptimizationOptimize key-value cache performance through:PagingPrefix cachingCache-aware memory layout designReduce memory pressure while sustaining high throughputLatency & Throughput OptimisationDesign and tune:Batching strategiesContinuous batchingSpeculative decodingOptimize tail latency and tokens/sec under real production traffic patternsWhat Success Looks LikeModels deploy efficiently on edge and embedded devices, not just serversTokens/sec significantly outperform baseline implementationsEnd-to-end latency is minimized and predictableInference cost per request is materially reducedThe company is no longer dependent on partners for inference optimizationRequired Experience & SkillsStrongly RequiredProven experience optimizing ML inference performance in productionDeep understanding of GPU architecture and memory hierarchiesHands-on experience with CUDA and low-level performance tuningExperience deploying models beyond research environmentsCritical Technical SkillsInference engines: vLLM, TensorRT-LLM, llama.cpp, QAIRTCUDA kernel development and profilingQuantisation techniques: INT8/INT4/FP4/FP8, AWQ, GPTQKV cache optimisation and memory layout designLatency optimisation: batching, speculative decoding, continuous batchingCommon Problems You'll Be SolvingDeploy efficiently on edge or embedded targetsAchieve competitive tokens/secReduce and stabilize inference latencyYou will be responsible for closing these gaps, creating a major competitive advantage.What we offerWe offer a generous compensation and benefits package (in addition to the base salary), including:Salary range $185,000.00 USD - $280,000.00 USD It is not typical for offers to be made at or near the top of the range. The actual salary will be determined based on experience and other job-related factors.Annual bonus opportunityInsurance coverage (medical, dental, vision, life, and disability)Paid time offPaid holidaysCompany contribution to the 401KEquity awards for certain positions and levelsRemote and/or hybrid work available depending on the positionAll compensation and benefits are subject to the terms and conditions of the underlying plans or programs, as applicable, and may be amended, terminated, or replaced from time to time.Cerence Inc. (Nasdaq: CRNC and www.cerence.com) is the global industry leader in creating unique, moving experiences for the automotive world. Spun out from Nuance in October 2019, Cerence is a new, independent company that has quickly gained traction as a leader in the automotive voice assistant space, working with all of the world's leading automakers - from Ford and Fiat Chrysler to Daimler, Audi and BMW to Geely and SAIC - to transform how a car feels, responds and learns. Its track record is built on more than 20 years of industry experience and leadership and more than 500 million cars on the road today across more than 70 languages.As Cerence looks to the future and continues an ambitious growth agenda, we need someone to join the team and help build the future of voice and AI in cars. This is an exciting opportunity to join Cerence's passionate, dedicated, global team and be a part of meaningful innovation in a rapidly growing industry.EQUAL OPPORTUNITY EMPLOYERCerence is firmly committed to Equal Employment Opportunity (EEO) and to compliance with all federal, state and local laws that prohibit employment discrimination on the basis of age, race, color, gender, gender identity, gender expression, sex, sex stereotyping, pregnancy, national origin, ancestry, religion, physical or mental disability, medical condition, marital status, citizenship status, sexual orientation, protected military or veteran status, genetic information and other protected classifications. Cerence Equal Employment Opportunity Policy Statement.All prospective and current Employees need to remain vigilant when it comes to executing security policies in the workplace. This includes:Following workplace security protocols and training programs to familiarize with the ways to maintain a safe workplace.Following security procedures to report any suspicious activity.Having respect for corporate security procedures to allow those procedures to be effective.Adhering to company's compliance and regulations.Encouraging to follow a zero tolerance for workplace violence.Basic knowledge of information security and data privacy requirements (e.g., how to protect data & how to be handling this data).Demonstrative knowledge of information security through internal training programs.