Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Inference
Senior Software Engineer - Model Performance
Inference
San Francisco•Hybrid••2-5 Yrs•3 hours ago
Job description
Hybrid
Overview
Help us make inference blazingly fast. If you love squeezing every last drop of performance out of GPUs, diving deep into CUDA kernels, and turning optimization techniques into production systems, we'd love to meet you.
Responsibilities
- Implement and productionize optimization techniques including quantization, speculative decoding, KV cache optimization, continuous batching, and LoRA serving
- Deep dive into inference frameworks (vLLM, SGLang, TensorRT-LLM) and underlying libraries to debug and improve performance
- Profile and optimize CUDA kernels and GPU utilization across our serving infrastructure
- Add support for new model architectures, ensuring they meet our performance standards before going to production
- Experiment with novel inference techniques and bring successful approaches into production
- Build tooling and benchmarks to measure and track inference performance across our fleet
- Collaborate with applied ML engineers to ensure trained models can be served efficiently
Qualifications
- 2+ years of experience in ML systems, inference optimization, or GPU programming
- Strong proficiency in Python and familiarity with C++
- Hands-on experience with LLM inference frameworks (vLLM, SGLang, TensorRT-LLM, or similar)
- Deep understanding of GPU architecture and experience profiling GPU workloads
- Familiarity with LLM optimization techniques (quantization, speculative decoding, continuous batching, KV cache management)
- Track record of measurably improving system performance
- Familiarity with serving non-LLM models (TTS, vision, embeddings)
- Contributions to open-source inference frameworks
- You don't need to tick every box. Curiosity and the ability to learn quickly matter more.
- Compensation
- We offer competitive compensation, equity in a high-growth startup, and comprehensive benefits. The base salary range for this role is $220,000 - $320,000, plus equity and benefits, depending on experience.
- Equal Opportunity
- Inference.net is an equal opportunity employer. We welcome applicants from all backgrounds and don't discriminate based on race, color, religion, gender, sexual orientation, national origin, genetics, disability, age, or veteran status.
- If you're excited about making AI inference faster for everyone, we'd love to hear from you. Please send your resume and GitHub to amar@inference.net and/or apply here on Ashby.