Skip to content

// cloudmash.blog

Software internals, written down.

Long-form companions to the Cloudmash videos. Each post follows one request, event or byte through a real system and shows what every component does with it.

// latest posts

20 min read + video

Your GPU holds 24 GB. This model is 94 GB. It still runs.

llama.cpp's --moe-cache-mib keeps MoE experts in RAM and caches the hot ones in VRAM: a 94 GB model on a 24 GB RTX 4090, with each MoE flag explained.

AI infrastructureLLM inferenceGPUsinternals