Browse Jobs
Explore open roles across India.Explore active software roles. Sign up to apply and get AI match scores.
Top MatchesTop Job Matches
Loading jobs…
Back to Browse Jobs
Plaud
Machine Learning Engineer, Model Evaluations (Speech LLM) - San Francisco
Plaud
San Francisco•Hybrid•Salary•3-5 Yrs•1 hour ago
Job description
Speech LLMAI work companionNote-taking solutionsHuman-AI interactionML evaluation
Overview
Machine Learning Engineer, Model Evaluations (Speech LLM) in San Francisco. Role involves defining metrics for speech LLM performance, building evaluation frameworks, and tracking model health.
Responsibilities
- Define automated metrics from subjective concepts
- Build distributed systems and data pipelines for model evaluation at scale
- Partner with researchers to translate capabilities into measurable benchmarks
- Develop dashboards that track model health during training
- Debug performance issues in mid-training runs
- Communicate statistical findings to both technical and non-technical audiences
Qualifications
- Passion for turning ambiguous concepts into quantitative metrics
- Strong software engineering skills (especially Python)
- Experience building reliable distributed systems and data pipelines
- Experience with model evaluation frameworks
- Ability to collaborate with ML researchers
- Rapid debugging of training anomalies
- Clear communication of complex statistical findings
Nice to have
- Speech Metrics: WER, CER, PESQ, automated MOS scoring
- LLM-as-a-Judge for evaluating conversational logic, transcription accuracy, audio quality, and reasoning
- Human evaluation via crowdsourcing or preference data collection
- Observability with statistics and experimental design background
- Adversarial datasets for testing edge cases
Benefits
- Competitive compensation ($200K -$365K base + performance bonus + equity)
- Top-tier healthcare (including dental and vision)
- 401(k) plan with company matching
- Unlimited PTO plus 13 paid holidays
- 12 weeks paid parental leave
- Hybrid office model (minimum 3 days in-office per week)
- Choice of high-end laptops/workstations
- Annual offsites and a fully stocked office