Skip to content
NerdLevelTech
Intermediate

LLM Production Infrastructure: High-Performance Serving & Optimization

Master LLM inference optimization with vLLM, TensorRT-LLM, and production observability. Learn KV cache management, speculative decoding, LLM gateways, and cost optimization strategies for enterprise-scale AI deployments.

76 min22 lessons6 modulesDevOps

What you'll learn

  • Optimize LLM inference performance using vLLM and TensorRT-LLM
  • Implement KV cache management for efficient memory utilization
  • Apply speculative decoding techniques for faster token generation
  • Design and deploy LLM gateways with LiteLLM for multi-provider routing
  • Implement cost optimization strategies for enterprise-scale deployments

Course Content

  1. LLM Inference Fundamentals

    4 lessons · 14 min
    • The LLM Inference PipelineNo signup4 min
    • Locked until you enrollKV Cache: The Memory Optimization Foundation4 min
    • Locked until you enrollBatching Strategies for Maximum Throughput3 min
    • Locked until you enrollSpeculative Decoding for Faster Generation3 min
    • Quiz: Module 1: LLM Inference Fundamentals

  2. vLLM & Open-Source Inference Engines

    4 lessons · 14 min
    • Locked until you enrollvLLM Architecture & Core Concepts4 min
    • Locked until you enrollvLLM Production Deployment4 min
    • Locked until you enrollAlternative Inference Engines: TGI, OpenLLM & SGLang3 min
    • Locked until you enrollPrefix Caching & Advanced Optimization3 min
    • Quiz: Module 2: vLLM & Open-Source Inference Engines

  3. TensorRT-LLM & GPU Optimization

    4 lessons · 14 min
    • Locked until you enrollTensorRT-LLM Architecture & Setup4 min
    • Locked until you enrollLLM Quantization Techniques4 min
    • Locked until you enrollBlackwell GPU Optimization3 min
    • Locked until you enrollTensorRT-LLM Performance Tuning3 min
    • Quiz: Module 3: TensorRT-LLM & GPU Optimization

  4. Production Observability

    4 lessons · 14 min
    • Locked until you enrollLLM Observability Fundamentals4 min
    • Locked until you enrollLangfuse: Open-Source LLM Observability4 min
    • Locked until you enrollHelicone: High-Performance LLM Proxy3 min
    • Locked until you enrollLangSmith: LangChain's Observability Platform3 min
    • Quiz: Module 4: Production Observability

  5. LLM Gateways & Routing

    3 lessons · 10 min
    • Locked until you enrollLLM Gateway Architecture3 min
    • Locked until you enrollLiteLLM: Unified LLM Gateway4 min
    • Locked until you enrollModel Routing Strategies3 min
    • Quiz: Module 5: LLM Gateways & Routing

  6. Cost Optimization & Scaling

    3 lessons · 10 min
    • Locked until you enrollLLM Cost Analysis3 min
    • Locked until you enrollCost Optimization Strategies4 min
    • Locked until you enrollScaling Patterns3 min
    • Quiz: Module 6: Cost Optimization & Scaling