{"schemaVersion":"jobsearcher.job.v1","id":"0dd54dace129cc8d2af3b5d0","url":"https://jobsearcher.com/jobs/0dd54dace129cc8d2af3b5d0","canonicalUrl":"https://jobsearcher.com/jobs/0dd54dace129cc8d2af3b5d0","title":"Software Engineer (Gen AI Solutions)","description":"HomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsHomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsJob DescriptionJob OverviewJob ID:J55232Job Title:Software Engineer (Gen AI Solutions)Location:Austin, TXDuration:14 Months + ExtensionDomain Exposure:PharmaceuticalsClient:To Be Discussed LaterEmployment Type:W-2, 1099, C2CWork Authorization:US Citizen, Green Card, H-1B, OPT-EAD, CPT, OtherView All JobsSimilar Jobs Software Engineer - ALBANY, NY Principal Java Backend Softwar... - Cincinnati, OH Software Lead Engineer - Plano, TX Staff Software Engineer - Austin, TX Software Lead Engineer - Plano, TXJob Title: Software Engineer (Gen AI Solutions)Location: Austin, TXDuration: 14 Months + ExtensionBill Rate: $76/hour (C2C)Job Type: W-2, 1099, C2CClient: To Be Discussed LaterWork Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, OtherJob DescriptionWe need an experienced software engineer to help design, build, and optimize production LLM inference services that power GenAI features for internal and customer-facing applications. The engineer will work with our ML, platform, and product teams to make model serving faster, cheaper, and more reliable at scale.Scope Of WorkDesign, deploy, and maintain scalable inference services for open-weight and hosted LLMs on Kubernetes-based infrastructure.Profile and reduce end-to-end latency (time to first token, tokens per second) and cost per request.Apply model and serving optimizations, then benchmark the trade-offs between quality, latency, and throughput.Build and maintain integrations with external model APIs and internal orchestration frameworks, including routing, fallback, retries, and rate-limit handling.Add observability across the request path, covering logging, metrics, tracing, and dashboards for request volume, downstream call ratios, errors, and token usage.Set up evaluation and regression pipelines so model or serving changes don't degrade output quality.Do capacity planning and autoscaling, and track GPU utilization and cost.Write technical design docs and runbooks, and hand knowledge over to the internal team.Expected DeliverablesProduction-ready inference service(s) with CI/CD and infrastructure as codeLatency and cost benchmark reports, measured before and after optimizationMonitoring dashboards and alertingArchitecture docs, runbooks, and handover sessionsRequiredSoftware engineering experience, including 2+ years on ML/LLM systems in productionStrong Python, plus working knowledge of Go, C++, or Rust for performance-critical pathsHands-on experience with LLM serving frameworks: vLLM, TensorRT-LLM, TGI, SGLang, or Triton Inference ServerInference optimization techniques:Quantization (INT8/INT4, FP8, AWQ, GPTQ)KV-cache management, paged attention, prefix cachingContinuous/dynamic batching and speculative decodingTensor and pipeline parallelismExperience with PyTorch and the Hugging Face ecosystemKubernetes, Docker, and GPU scheduling (NVIDIA GPU operator, node pools, autoscaling)Integration with hosted LLM APIs (OpenAI, Gemini, Anthropic), including streaming, retries, and rate limitingObservability tooling: Prometheus/Grafana, Splunk, Datadog, OpenTelemetrySolid grounding in distributed systems, API design (REST/gRPC), and performance profilingPreferredGenAI orchestration frameworks: Genkit, LangChain, LlamaIndexCUDA, Triton kernels, FlashAttention, or other low-level GPU optimizationModel compilation and runtimes: ONNX Runtime, TorchScript/torch.compile, Core MLRAG pipelines and vector databases (FAISS, Milvus, pgvector)LLM evaluation frameworks and building automated quality and regression benchmarksCost modeling and FinOps for GPU workloadsInfrastructure as code (Terraform, Helm) and CI/CD (GitHub Actions, Jenkins)Contributions to open-source inference projects, or published work in ML systemsSoft SkillsCan turn ambiguous requirements into concrete technical plans and trade-off proposalsCommunicates clearly in writing (design docs, benchmark reports)Works well with cross-functional teams of engineers and PMs.Apply NowEqual Opportunity EmployerWe are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.Home|About|Adaptive AI|Industries|Consulting |Contact Us© 2004-2026, Machine Learning Technologies LLC, or its affiliates","company":"Machine Learning Technologies","rawCompany":"machine learning technologies","city":"Austin","state":"TX","isRemote":false,"isActive":false,"createdAt":"2026-11-14T08:23:07.963Z","occupations":[{"code":"15-1252.00","title":"Software Developers","slug":"software-developers"},{"code":"15-1299.08","title":"Computer Systems Engineers/Architects","slug":"computer-systems-engineers-architects"},{"code":"15-1251.00","title":"Computer Programmers","slug":"computer-programmers"}],"industries":[{"code":"541511","title":"Custom Computer Programming Services","slug":"custom-computer-programming-services"},{"code":"513210","title":"Software Publishers","slug":"software-publishers"},{"code":"541512","title":"Computer Systems Design Services","slug":"computer-systems-design-services"}],"jobPosting":{"@context":"https://schema.org","@type":"JobPosting","title":"Software Engineer (Gen AI Solutions)","description":"HomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsHomeAboutAdaptive AIIndustriesConsulting Jobs PortalContact UsJob DescriptionJob OverviewJob ID:J55232Job Title:Software Engineer (Gen AI Solutions)Location:Austin, TXDuration:14 Months + ExtensionDomain Exposure:PharmaceuticalsClient:To Be Discussed LaterEmployment Type:W-2, 1099, C2CWork Authorization:US Citizen, Green Card, H-1B, OPT-EAD, CPT, OtherView All JobsSimilar Jobs Software Engineer - ALBANY, NY Principal Java Backend Softwar... - Cincinnati, OH Software Lead Engineer - Plano, TX Staff Software Engineer - Austin, TX Software Lead Engineer - Plano, TXJob Title: Software Engineer (Gen AI Solutions)Location: Austin, TXDuration: 14 Months + ExtensionBill Rate: $76/hour (C2C)Job Type: W-2, 1099, C2CClient: To Be Discussed LaterWork Authorization: US Citizen, Green Card, OPT-EAD, CPT, H-1B, H4-EAD, OtherJob DescriptionWe need an experienced software engineer to help design, build, and optimize production LLM inference services that power GenAI features for internal and customer-facing applications. The engineer will work with our ML, platform, and product teams to make model serving faster, cheaper, and more reliable at scale.Scope Of WorkDesign, deploy, and maintain scalable inference services for open-weight and hosted LLMs on Kubernetes-based infrastructure.Profile and reduce end-to-end latency (time to first token, tokens per second) and cost per request.Apply model and serving optimizations, then benchmark the trade-offs between quality, latency, and throughput.Build and maintain integrations with external model APIs and internal orchestration frameworks, including routing, fallback, retries, and rate-limit handling.Add observability across the request path, covering logging, metrics, tracing, and dashboards for request volume, downstream call ratios, errors, and token usage.Set up evaluation and regression pipelines so model or serving changes don't degrade output quality.Do capacity planning and autoscaling, and track GPU utilization and cost.Write technical design docs and runbooks, and hand knowledge over to the internal team.Expected DeliverablesProduction-ready inference service(s) with CI/CD and infrastructure as codeLatency and cost benchmark reports, measured before and after optimizationMonitoring dashboards and alertingArchitecture docs, runbooks, and handover sessionsRequiredSoftware engineering experience, including 2+ years on ML/LLM systems in productionStrong Python, plus working knowledge of Go, C++, or Rust for performance-critical pathsHands-on experience with LLM serving frameworks: vLLM, TensorRT-LLM, TGI, SGLang, or Triton Inference ServerInference optimization techniques:Quantization (INT8/INT4, FP8, AWQ, GPTQ)KV-cache management, paged attention, prefix cachingContinuous/dynamic batching and speculative decodingTensor and pipeline parallelismExperience with PyTorch and the Hugging Face ecosystemKubernetes, Docker, and GPU scheduling (NVIDIA GPU operator, node pools, autoscaling)Integration with hosted LLM APIs (OpenAI, Gemini, Anthropic), including streaming, retries, and rate limitingObservability tooling: Prometheus/Grafana, Splunk, Datadog, OpenTelemetrySolid grounding in distributed systems, API design (REST/gRPC), and performance profilingPreferredGenAI orchestration frameworks: Genkit, LangChain, LlamaIndexCUDA, Triton kernels, FlashAttention, or other low-level GPU optimizationModel compilation and runtimes: ONNX Runtime, TorchScript/torch.compile, Core MLRAG pipelines and vector databases (FAISS, Milvus, pgvector)LLM evaluation frameworks and building automated quality and regression benchmarksCost modeling and FinOps for GPU workloadsInfrastructure as code (Terraform, Helm) and CI/CD (GitHub Actions, Jenkins)Contributions to open-source inference projects, or published work in ML systemsSoft SkillsCan turn ambiguous requirements into concrete technical plans and trade-off proposalsCommunicates clearly in writing (design docs, benchmark reports)Works well with cross-functional teams of engineers and PMs.Apply NowEqual Opportunity EmployerWe are an equal opportunity employer. All aspects of employment including the decision to hire, promote, discipline, or discharge, will be based on merit, competence, performance, and business needs. We do not discriminate on the basis of race, color, religion, marital status, age, national origin, ancestry, physical or mental disability, medical condition, pregnancy, genetic information, gender, sexual orientation, gender identity or expression, national origin, citizenship/ immigration status, veteran status, or any other status protected under federal, state, or local law.Home|About|Adaptive AI|Industries|Consulting |Contact Us© 2004-2026, Machine Learning Technologies LLC, or its affiliates","datePosted":"2026-11-14T08:23:07.963Z","dateModified":"2026-11-14T08:23:07.963Z","hiringOrganization":{"@type":"Organization","name":"Machine Learning Technologies","sameAs":"https://jobsearcher.com"},"jobLocation":{"@type":"Place","address":{"@type":"PostalAddress","addressLocality":"Austin","addressRegion":"TX","addressCountry":"US"}},"identifier":{"@type":"PropertyValue","name":"JobSearcher","value":"0dd54dace129cc8d2af3b5d0"},"url":"https://jobsearcher.com/jobs/0dd54dace129cc8d2af3b5d0"}}