Tag

#llm-inference

3 posts

AI & Machine LearningMay 2, 202611 min
FlashKDA: Moonshot's CUTLASS Kernel for Kimi Linear (2026)
Moonshot open-sourced FlashKDA, a CUTLASS CUDA kernel for Kimi Delta Attention. Drop-in for flash-linear-attention with up to 2.22x prefill speedup on H20 GPUs.
AI & Machine LearningApril 6, 202611 min
Google's TurboQuant: 6x Less Memory for LLM Inference (2026)
Google's TurboQuant compresses LLM KV caches to 3 bits with zero accuracy loss, cutting memory 6x and speeding up H100 attention computation up to 8x vs FP32.
AI & Machine LearningApril 5, 202611 min
AirLLM Tested: Run a 70B LLM on a 4GB GPU — Does It Work?
Run a 70B LLM on a 4GB GPU? AirLLM uses layer-wise inference, no quantization. Benchmarks, latency tradeoffs, and how it compares to Ollama + llama.cpp.

Every post in #llm-inference (3)

FlashKDA: Moonshot's CUTLASS Kernel for Kimi Linear (2026)