Overview

Our Engineering organisation is scaling its AI Engineering capabilities under new leadership, creating an opportunity to build the next generation of AI at Rapid7.

Responsibilities

• Define the architecture and technical roadmap for LLM and agentic AI systems across Detection and Response and Exposure and Remediation
• Lead model strategy: when to use frontier or open-source models, and when to fine-tune, distill, or post-train
• Fine-tune, distill and post-train open-source LLMs on security data, from data curation through training, evaluation and deployment
• Set engineering standards for context engineering, inference optimization, and evaluation across teams
• Optimize LLM inference at scale with techniques such as speculative decoding, KV-cache management, continuous batching, quantization and custom kernels, balancing quality, latency and cost
• Establish evals as the release gate for every AI system we ship
• Lead design reviews and resolve the hardest technical trade-offs
• Stay hands-on: prototype and build critical model, training and inference code yourself
• Mentor Staff and Senior engineers and shape product and platform investment
• 12+ years of software or ML engineering experience, including 2+ years building production LLM or agentic AI systems. A PhD in Computer Science, Machine Learning, AI or a related field is preferred, or equivalent depth of expertise demonstrated through industry experience and technical contributions
• A track record of building scalable, production-grade AI systems used across multiple teams or products with a strong history of organization-wide impact through technical leadership, mentoring, and shared platforms and standards
• LLMs from first principles: deep understanding of transformer architecture, attention, tokenization and positional encodings, and how models are trained and served, with the ability to implement core components from scratch
• PyTorch: expert-level, including custom model code, training loops, mixed precision and distributed training (e.g., FSDP, DeepSpeed)
• Fine-tuning and post-training: hands-on full and parameter-efficient fine-tuning (LoRA, QLoRA) of open-weight LLMs, distillation, and preference or RL-based methods (e.g., DPO, GRPO)
• Context engineering: retrieval, memory, and context management for long-running agent workflows
• Inference optimization: deep knowledge of prefill and decode, KV-cache management (e.g., PagedAttention, prefix caching), continuous batching, speculative decoding, FlashAttention and quantization, and serving at scale with vLLM, SGLang or TensorRT-LLM
• Evals: defining evaluation strategy and benchmarks for LLM and agent systems
• Strong predictive AI and classical ML fundamentals, and the judgment to choose between them and LLMs based on accuracy, latency and cost
• Strong distributed systems and Python fundamentals
• Takes ownership of significant technical outcomes across multiple teams, proactively identifying and addressing risks, driving complex challenges to resolution and ensuring solutions deliver against agreed technical and customer objectives
• Strong communicator with the ability to explain technical trade-offs to engineering, product, and leadership audiences
• Builds strong cross functional partnerships across Engineering, Product, and other teams to align on technical direction and deliver shared outcomes
• Connects customer needs and business priorities to technical decisions, evaluating the value, cost and trade-offs of AI solutions to ensure investments deliver measurable customer and business outcomes
• Embody our core values to foster a culture of excellence that drives meaningful impact and collective success

Nice to have

• Experience pre-training or training LLMs from scratch
• Experience writing GPU kernels (CUDA or Triton) and profiling GPU workloads
• Security domain experience, such as detection engineering, SOC/MDR, or vulnerability management
• Experience running AI workloads on AWS at scale
• Publications, patents, or open-source contributions in AI