JOBSEARCHER

Speculative Decoding Engineer

ML Engineer - Speculative Decoding / LLM Inference / Model Distillation Mountain View, California, United States (On-site)I'm working with an early-stage AI infrastructure company developing software to improve LLM inference efficiency, reducing compute costs, latency and energy consumption. They're looking for an ML Engineer specialising in Speculative Decoding with experience in model training, distillation and inference optimisation.
The role: Train and distil smaller draft models to accelerate inference for larger LLMs. Develop PyTorch training pipelines and generate distillation datasets. Integrate draft models into vLLM, SGLang or Hugging Face serving frameworks. Optimise draft acceptance rates, verification overhead and inference throughput. Evaluate quantization, tokenization and model compatibility across open-weight LLMs. What they're looking for:2+ years of experience in ML engineering, model training or distillation. Strong hands-on experience with PyTorch and LLM model development. Understanding of speculative decoding, model compression and inference optimisation. Experience with vLLM, SGLang, Hugging Face or similar frameworks. Familiarity with open-weight models such as Llama, Qwen, DeepSeek or Mistral. Ability to evaluate model performance through measurable latency and throughput improvements. Why consider it? Join an early-stage AI company tackling the growing compute and energy demands of AI.Work directly with founders from Google TPU and Tesla Dojo. Own core model development and inference optimisation within a small founding team. Help make LLM inference faster, cheaper and more energy-efficient.
Location: Mountain View / South Bay, California - On-site. Interested? Apply directly or message me for a confidential discussion.