Senior Software Engineer
For over 16 years, Trilyon has been at the forefront of providing comprehensive global workforce solutions and staffing services. Leveraging our extensive expertise across multiple domains such as Cloud technology, Salesforce, AI, Machine Learning, and Technical Writing, we consistently exceed expectations in catering to a wide range of requirements. Currently we are seeking a “Software Development Engineer 5” for one of our clients that is a leading multination corporation.
Job Title: Software Development Engineer 5Location: Remote
Contract Term: 6 months contract
Position Overview
Must have:
deep kernel engineering expertise. Client will teach them agents but people who have familiarity with CuteDSL, FlyDSL, Triton, Tensile -- moving more towards a scheduler optimization agent rather than debugging.
About the role:
Client is building agentic software to improve how teams develop, deploy, and operate machine learning workloads on client GPUs. This senior contract role focuses on designing and delivering production-grade intelligent agents across the client software stack: training and inference frameworks, cluster tooling, performance analysis, and operational workflows for large-scale GPU deployments. You will lead implementation of Python systems that combine LLM orchestration, tool integration, and rigorous evaluation to automate high-value engineering work: investigating distributed job failures, interpreting telemetry and profilers, accelerating performance analysis, and reducing manual toil in ML operations. This is a senior individual contributor engagement: you are expected to own significant components end to end, make sound technical decisions within program direction, and deliver maintainable code with minimal supervision.
Key responsibilities:
Architect and implement Python agent frameworks: tool adapters, orchestration loops, structured outputs, session handling, and CLI or service interfaces suitable for production use. Build LLM-powered workflows (LiteLLM, DSPy, or equivalent) with bounded prompts, allowlisted tools, budget caps, structured JSON outputs, and evidence-backed reporting. Integrate agents with the client GPU and ML software ecosystem: ROCm tooling, PyTorch, distributed training and inference stacks, profilers, cluster metrics, log pipelines, and debug utilities. Own quality and eval infrastructure: fixture-driven tests, JSON Schema validation, regression suites, accuracy metrics, and CI integration before rollout. Solve ML operations problems on training and inference clusters: multi-node failure analysis, performance regression investigation, configuration and launch issues, and operational automation. Deliver deployment-ready artifacts: Docker / Kubernetes packaging, runbooks, documentation, and security-conscious defaults for customer-controlled environments. Partner with framework, performance, and field teams; incorporate code review feedback and iterate based on production and pilot learnings.
Required qualifications:
Bachelor’s degree or higher in Computer Science, Computer Engineering, Electrical Engineering, or a related field; equivalent practical experience accepted.10+ years of professional software engineering experience, including 5+ years on production Python systems at scale. Deep experience with distributed ML training or inference on GPU clusters: multi-node jobs, collective communication failures, log and metric correlation across ranks and nodes. Strong hands-on PyTorch background and production familiarity with large-scale stacks (Megatron-LM, DeepSpeed, TorchTitan, vLLM, or equivalent).Proven ML debugging and performance engineering across multiple failure modes: throughput regression, memory errors, numerical instability, misconfiguration, and profiler or trace analysis. Demonstrated delivery of LLM agent or orchestration systems in production or near-production settings: tool routing, structured outputs, reliability, and testability. Track record of shipping complex software on schedule: clean code, automated tests, code review discipline, and clear technical communication. Ability to work independently, prioritize across ambiguous requirements, and align weekly with a technical lead.
Preferred qualifications:
Open-source ML development: upstream framework repos, community CI patterns, and integration with OSS tooling.
ML performance optimization: parallelism, throughput/MFU tuning, roofline analysis, and profiler-driven investigation.
ROCm and GPU cluster operations: metrics, profilers, health monitoring, Slurm or Kubernetes job environments.
Security-aware agent deployment: secret handling, egress control, and customer VPC or on-prem constraints.
Equal Employment Opportunity:
Trilyon is an Equal Opportunity Employer, committed to fairness and respect for all individuals. We value diversity in age, disability, ethnicity, gender, gender identity, religion, and sexual orientation, believing it drives innovation and better service. Employment decisions are made impartially, without regard to any protected characteristic under federal, state, or local law. Our diverse team drives innovation, competitiveness, and creativity, enhancing our ability to effectively serve our clients and communities. This commitment to diversity makes us stronger and more adaptable. Warm Regards, Signature