{"schemaVersion":"jobsearcher.job.v1","id":"221aa85d71b1cb6101dd0dca","url":"https://jobsearcher.com/jobs/221aa85d71b1cb6101dd0dca","canonicalUrl":"https://jobsearcher.com/jobs/221aa85d71b1cb6101dd0dca","title":"LLM Inference Engineer","description":"LLM Inference EngineerEmployment Type: Full-Time (FTE)Team: Founding TeamExperience Level: New Graduate to SeniorLocation: Remote -SFO, CACompensation and BenefitsBase Salary: $120,000–$200,000Equity: 0.25%–2.0%, with four-year vesting and a one-year cliffCompensation offers flexibility to trade off base salary for additional equity.Benefits include health, dental, vision, and flexible time off.Role OverviewWe are looking for an Inference Engineer to optimize the performance and cost of large language model (LLM) serving engines, including vLLM and SGLang. The role focuses on profiling inference workloads, improving GPU memory utilization, optimizing batching and caching, developing GPU kernels, and benchmarking performance on both NVIDIA and AMD GPUs.The ideal candidate combines strong systems engineering and GPU programming skills with hands-on experience modifying inference engines and demonstrating measurable performance improvements. Candidates ranging from new graduates to senior engineers are welcome, including those with relevant research, internships, thesis work, or open-source contributions.Key ResponsibilitiesProfile LLM inference workloads to identify performance bottlenecks involving execution time, GPU memory, latency, and throughput.Develop and optimize inference-serving mechanisms, including continuous batching, KV cache, prefix caching, PagedAttention, speculative decoding, and chunked prefill.Optimize scheduling and GPU resource utilization, including workload distribution across GPUs and prefill/decode execution.Develop and optimize GPU kernels and runtime components using CUDA, Triton, ROCm, HIP, and related technologies.Optimize model serving across NVIDIA and AMD GPU architectures.Benchmark inference performance using metrics such as time-to-first-token (TTFT), inter-token latency (ITL/TPOT), tokens per second, throughput, and p99 latency.Use profiling tools, hardware measurements, benchmark harnesses, and load testing to identify bottlenecks and validate improvements.Support quantization and memory optimization techniques, including FP8, INT4, AWQ, and GPTQ.Bring newly released open-source models into the serving environment and optimize their inference performance.Contribute production-ready code and improvements to open-source projects such as vLLM, SGLang, and related inference infrastructure.Required QualificationsStrong background in Computer Science, Engineering, Machine Learning, Systems, or a related field.Hands-on experience working with LLM inference or serving engines, such as vLLM, SGLang, TensorRT-LLM, llama.cpp, TGI, or FlashInfer.Understanding of inference-serving internals, including KV cache, continuous batching, prefix caching, PagedAttention, speculative decoding, and prefill/decode execution.Experience with performance profiling, benchmarking, and quantitative evaluation of inference workloads.Strong systems programming skills in C++, Rust, CUDA, or Python, with an understanding of distributed systems, concurrency, memory management, schedulers, compilers, or high-performance computing (HPC).Experience with GPU programming and kernel optimization using technologies such as CUDA, Triton, ROCm, HIP, or CUTLASS.Ability to investigate performance bottlenecks, implement optimizations, and demonstrate measurable results.Experience contributing code to production systems or open-source projects is valuable.Preferred QualificationsExperience optimizing inference on NVIDIA H100/A100 or AMD MI300X/MI250 GPUs.Knowledge of quantization techniques, including FP8, INT4, AWQ, and GPTQ.Experience with mixture-of-experts (MoE) serving and distributed inference.Contributions to open-source projects, including vLLM, SGLang, TensorRT-LLM, or related GPU and compiler projects.Experience with profiling tools such as Nsight, PyTorch Profiler, or rocprof.Publications or research in systems and machine learning venues such as MLSys, OSDI, SOSP, or ASPLOS.Experience developing benchmark infrastructure, load-testing systems, or performance regression detection.Relevant internships, thesis projects, research papers, or personal GitHub projects demonstrating inference optimization skills.Key SkillsLLM Inference | vLLM/SGLang | CUDA/ROCm | GPU Kernel Optimization | Performance Profiling | KV Cache","company":"Mindsource","rawCompany":"mindsource","city":"California","state":"MO","isRemote":false,"isActive":false,"createdAt":"2026-10-30T07:48:42.757Z","occupations":[{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1221.00","title":"Computer and Information Research Scientists","slug":"computer-and-information-research-scientists"}],"industries":[{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"},{"code":"518210","title":"Computing Infrastructure Providers, Data Processing, Web Hosting, and Related Services","slug":"computing-infrastructure-providers-data-processing-web-hosting-and-related-services"},{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"LLM Inference Engineer","description":"LLM Inference EngineerEmployment Type: Full-Time (FTE)Team: Founding TeamExperience Level: New Graduate to SeniorLocation: Remote -SFO, CACompensation and BenefitsBase Salary: $120,000–$200,000Equity: 0.25%–2.0%, with four-year vesting and a one-year cliffCompensation offers flexibility to trade off base salary for additional equity.Benefits include health, dental, vision, and flexible time off.Role OverviewWe are looking for an Inference Engineer to optimize the performance and cost of large language model (LLM) serving engines, including vLLM and SGLang. The role focuses on profiling inference workloads, improving GPU memory utilization, optimizing batching and caching, developing GPU kernels, and benchmarking performance on both NVIDIA and AMD GPUs.The ideal candidate combines strong systems engineering and GPU programming skills with hands-on experience modifying inference engines and demonstrating measurable performance improvements. Candidates ranging from new graduates to senior engineers are welcome, including those with relevant research, internships, thesis work, or open-source contributions.Key ResponsibilitiesProfile LLM inference workloads to identify performance bottlenecks involving execution time, GPU memory, latency, and throughput.Develop and optimize inference-serving mechanisms, including continuous batching, KV cache, prefix caching, PagedAttention, speculative decoding, and chunked prefill.Optimize scheduling and GPU resource utilization, including workload distribution across GPUs and prefill/decode execution.Develop and optimize GPU kernels and runtime components using CUDA, Triton, ROCm, HIP, and related technologies.Optimize model serving across NVIDIA and AMD GPU architectures.Benchmark inference performance using metrics such as time-to-first-token (TTFT), inter-token latency (ITL/TPOT), tokens per second, throughput, and p99 latency.Use profiling tools, hardware measurements, benchmark harnesses, and load testing to identify bottlenecks and validate improvements.Support quantization and memory optimization techniques, including FP8, INT4, AWQ, and GPTQ.Bring newly released open-source models into the serving environment and optimize their inference performance.Contribute production-ready code and improvements to open-source projects such as vLLM, SGLang, and related inference infrastructure.Required QualificationsStrong background in Computer Science, Engineering, Machine Learning, Systems, or a related field.Hands-on experience working with LLM inference or serving engines, such as vLLM, SGLang, TensorRT-LLM, llama.cpp, TGI, or FlashInfer.Understanding of inference-serving internals, including KV cache, continuous batching, prefix caching, PagedAttention, speculative decoding, and prefill/decode execution.Experience with performance profiling, benchmarking, and quantitative evaluation of inference workloads.Strong systems programming skills in C++, Rust, CUDA, or Python, with an understanding of distributed systems, concurrency, memory management, schedulers, compilers, or high-performance computing (HPC).Experience with GPU programming and kernel optimization using technologies such as CUDA, Triton, ROCm, HIP, or CUTLASS.Ability to investigate performance bottlenecks, implement optimizations, and demonstrate measurable results.Experience contributing code to production systems or open-source projects is valuable.Preferred QualificationsExperience optimizing inference on NVIDIA H100/A100 or AMD MI300X/MI250 GPUs.Knowledge of quantization techniques, including FP8, INT4, AWQ, and GPTQ.Experience with mixture-of-experts (MoE) serving and distributed inference.Contributions to open-source projects, including vLLM, SGLang, TensorRT-LLM, or related GPU and compiler projects.Experience with profiling tools such as Nsight, PyTorch Profiler, or rocprof.Publications or research in systems and machine learning venues such as MLSys, OSDI, SOSP, or ASPLOS.Experience developing benchmark infrastructure, load-testing systems, or performance regression detection.Relevant internships, thesis projects, research papers, or personal GitHub projects demonstrating inference optimization skills.Key SkillsLLM Inference | vLLM/SGLang | CUDA/ROCm | GPU Kernel Optimization | Performance Profiling | KV Cache","datePosted":"2026-10-30T07:48:42.757Z","dateModified":"2026-10-30T07:48:42.757Z","hiringOrganization":{"@type":"Organization","name":"Mindsource","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"California","addressRegion":"MO","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"221aa85d71b1cb6101dd0dca"},"url":"https://jobsearcher.com/jobs/221aa85d71b1cb6101dd0dca"}}