JOBSEARCHER

Senior LLM Model Optimization Consultant

About the Role: We are seeking a highly experienced Senior LLM Model Optimization Engineer for a consulting engagement focused on optimizing Large Language Models (LLMs) deployed in an on-premises environment. The ideal candidate will have deep hands-on expertise in transformer architectures, neural network weight tuning, layer-level optimization, and GPU-accelerated deep learning. This role requires experience working directly with model internals to improve accuracy, computational efficiency, memory utilization, and inference performance. This is a model-level engineering role, not a prompt engineering, RAG, or LLM application development position.
Key Responsibilities: Analyze and optimize transformer-based LLM architectures, including attention mechanisms, feed-forward layers, and normalization components. Perform model weight tuning through full and selective fine-tuning, gradient-based optimization, and parameter-efficient techniques. Identify opportunities for layer-level optimization, including layer freezing, pruning, compression, and architectural improvements. Optimize model training and inference performance on on-premises GPU infrastructure. Apply techniques such as quantization, mixed-precision computation, knowledge distillation, and model compression. Profile and troubleshoot GPU memory utilization, computational bottlenecks, and model execution performance. Develop benchmarking and evaluation frameworks to measure accuracy, latency, throughput, and resource utilization. Collaborate with infrastructure and AI engineering teams to deploy and validate optimized models.
Required Qualifications: 5+ years of hands-on experience in machine learning or deep learning engineering, including substantial experience with transformer-based models. Strong expertise in Python, PyTorch, and transformer architectures. Demonstrated experience tuning LLM model weights, including optimizer configuration, backpropagation, gradient management, and fine-tuning strategies. Hands-on experience modifying or optimizing transformer layers, attention mechanisms, and neural network components. Experience with model optimization techniques such as LoRA, QLoRA, pruning, quantization, and selective layer fine-tuning. Strong understanding of GPU computing, CUDA fundamentals, and GPU memory optimization. Experience evaluating model performance and balancing accuracy, latency, memory usage, and computational efficiency. Experience working with open-weight LLMs such as Llama, Mistral, or Qwen.
Preferred Qualifications: Experience with NVIDIA GPU environments and CUDA/C++ kernel development. Experience with DeepSpeed, PyTorch FSDP, or distributed model training.
Knowledge of Flash: Attention, Triton, TensorRT-LLM, or vLLM.Experience with quantization-aware training, structured pruning, or knowledge distillation. Experience optimizing LLMs on enterprise-managed or on-premises GPU clusters. Experience with Hugging Face Transformers, PEFT, and model profiling tools.
Ideal Candidate: We are particularly interested in engineers who have directly modified, fine-tuned, or optimized LLM weights and transformer layers, rather than exclusively integrating pretrained models through APIs. Candidates should be able to demonstrate measurable improvements in model quality, GPU memory efficiency, inference latency, or computational performance from previous engagements.