Overview

We are looking for an experienced GPU DevOps / Infrastructure Setup Engineer to build, configure, deploy, and maintain GPU-enabled infrastructure for AI/ML and High-Performance Computing (HPC) workloads.

Responsibilities

• Set up, configure, and maintain GPU-enabled servers, compute nodes, and AI infrastructure.
• Build and deploy on-premises AI infrastructure and GPU compute clusters.
• Perform end-to-end GPU cluster bring-up, including provisioning, configuration, validation, and testing.
• Configure and manage NVIDIA Base Command Manager (BCM) for cluster deployment and management, where applicable.
• Install and configure operating systems, GPU drivers, CUDA Toolkit, libraries, and supporting software.
• Deploy and maintain GPU workloads on bare-metal servers, cloud platforms, and hybrid environments.
• Configure GPU-enabled Docker containers and Kubernetes workloads.
• Deploy and manage LLM applications using frameworks such as vLLM, Ollama, Hugging Face Text Generation Inference (TGI), or NVIDIA Triton.
• Configure and troubleshoot GPU hardware, driver compatibility, CUDA initialization, memory utilization, and workload execution issues.
• Support InfiniBand networking and high-speed interconnects for multi-GPU or multi-node environments.
• Automate infrastructure provisioning, configuration management, and deployment using appropriate scripting and automation tools.
• Develop and maintain CI/CD pipelines for infrastructure and application deployments.
• Monitor GPU utilization, memory, temperature, power consumption, system health, and performance.
• Implement monitoring, logging, alerting, health checks, and automated recovery using tools such as Prometheus and Grafana.
• Optimize GPU utilization, VRAM consumption, inference latency, throughput, and overall infrastructure performance.
• Troubleshoot Linux, networking, storage, container, hardware, and application deployment issues.
• Perform infrastructure upgrades, patching, maintenance, capacity planning, and resource scaling.
• Maintain technical documentation, installation procedures, configuration guides, and troubleshooting records.
• Collaborate with AI/ML, development, platform, and infrastructure teams to support production workloads.

Qualifications

• Hands-on experience with GPU server setup, configuration, and operations.
• Knowledge of GPU hardware, server components, PCIe, power requirements, cooling, and hardware compatibility.
• Experience with GPU models such as NVIDIA A100, H100, H200, L40S, RTX series, or equivalent GPU platforms.
• Strong Linux administration and troubleshooting skills.
• Experience installing and configuring GPU drivers, CUDA Toolkit, and supporting libraries.
• Ability to validate GPU detection, driver functionality, CUDA compatibility, and workload execution.
• Experience building or maintaining on-premises AI infrastructure.
• Understanding of GPU cluster provisioning, compute nodes, and cluster validation.
• Experience with NVIDIA Base Command Manager (BCM) is preferred.
• Experience deploying on-premises GPU infrastructure without BCM is also valuable.
• Knowledge of InfiniBand, RDMA, and high-speed networking for GPU clusters is an advantage.
• Experience with Docker and GPU-enabled containers.
• Kubernetes experience for GPU workload deployment and orchestration.
• Infrastructure automation using Bash, Python, Ansible, Terraform, or equivalent tools.
• Familiarity with CI/CD pipelines, Git, configuration management, and infrastructure monitoring.
• Experience deploying LLM applications on GPU-enabled cloud instances or on-premises GPU servers.
• Familiarity with vLLM, Ollama, Hugging Face TGI, NVIDIA Triton, or equivalent model-serving frameworks.
• Understanding of API-based model serving and GPU resource allocation.
• Experience troubleshooting GPU memory issues and optimizing inference performance.
• Familiarity with Prometheus, Grafana, NVIDIA SMI, or DCGM for GPU monitoring.
• 3+ years of experience in DevOps, SRE, platform engineering, or infrastructure engineering.
• At least 1–2 years of hands-on experience working with GPU environments.
• Demonstrated experience in at least one real-world GPU infrastructure or AI deployment project.
• Ability to explain the infrastructure architecture, GPU hardware used, individual responsibilities, deployment process, and technical issues resolved.
• Experience supporting production environments is preferred.
• End-to-end GPU cluster deployment using NVIDIA BCM.
• Hands-on InfiniBand configuration and troubleshooting.
• Bare-metal GPU server installation and configuration without Docker.
• Multi-GPU or multi-node cluster deployment.
• Production LLM serving and GPU performance optimization.