Senior AI Platform / LLM Infrastructure Engineer

Role :Senior AI Platform / LLM Infrastructure Engineer

Location : Charlotte, NC (Hybrid)

Bill Rate: Rate: $67/hr

We are hiring a Senior AI Platform Engineer to build and optimize on-prem LLM inference platforms. The role focuses on high-performance model serving, GPU workloads, and scalable ML infrastructure using modern inference frameworks and Kubernetes.

Must-Have Skills

• LLM Inference Frameworks: vLLM, TensorRT-LLM, Triton Inference Server, SGLang

• Model Optimization: Continuous Batching, Speculative Decoding, KV Cache / Prefix Caching, FP8 / AWQ / GPTQ

• Distributed/Parallel Systems: Tensor Parallelism

• Platform & Orchestration: Kubernetes, KServe, OpenShift AI, Helm / Operators

• GPU & Performance: CUDA, NCCL, MIG, GPU Orchestration (Run:AI)

• Monitoring: Prometheus, Grafana, ML Observability

• Programming: Python

• GenAI Tools: Arize AI, Claude (CoWork)

• Load / performance testing: GuideLLM, Locust

🔧 Key Responsibilities

• Build and manage LLM inference platforms on on-prem GPU infrastructure

• Optimize model performance using advanced inference techniques (batching, caching, quantization)

• Deploy and operate ML workloads on Kubernetes (KServe/OpenShift AI)

• Enable GPU scheduling and orchestration for large-scale workloads

• Implement monitoring and performance benchmarking frameworks

• Drive SRE practices for platform reliability and scalability (observability, incident handling)

• Collaborate with AI/ML teams to enable production-grade GenAI deployments