ML Software Engineer, Data Plane

Overview In this role you will own and optimize the inference data plane for a custom ML accelerator, enabling efficient large-model execution across hardware, memory, and data movement. You’ll work closely with cross-functional teams to bring models from validation to production, shaping high-performance kernels and integration with serving frameworks. You’ll scale and validate end-to-end architectures, building test and profiling tools to drive latency and throughput. This ground-up role offers impact across the full stack and opportunities to influence frontier-scale ML deployments. ResponsibilitiesDevelop and optimize compute kernels for a custom ML accelerator to achieve production-level LLM inference performanceImplement and validate end-to-end LLM architectures from PyTorch models to distributed execution on custom hardwareIntegrate custom accelerator backends into open-source serving frameworks (vLLM, PyTorch) including scheduler extensions, memory management, and model parallelismBuild and maintain test infrastructure for model correctness across CPU, GPU, simulator, and hardware targetsProfile and optimize inference workloads, identify bottlenecks, and drive latency/throughput improvements from simulation to hardware bring-upOwn features end-to-end from design through implementation, testing, and integration into the software stackContribute to CI/CD pipelines to gate model and kernel changes on correctness and performance regressions Key requirementsBachelor's degree or equivalent4+ years of full software development lifecycle experienceKnowledge of computer architecture, operating systems, and parallel computingStrong proficiency in C/C++Strong Linux systems knowledgeExperience developing compute kernels for GPUs, DSPs, or custom acceleratorsProven track record of owning and delivering complex software features end-to-end跨团队协作能力问题解决能力自我驱动与责任感C/C++Compute kernels for GPUs/DSPs/custom acceleratorsLinux systems programming