RTX 5090 vs RTX 4090 for local LLMs

Computed with the open FitLLM engine — accurate per-layer KV-cache modeling, not a naive estimate. Updated 2026-07-16.

FitLLM compares fit — what loads in memory — computed from official config.json. These are a floor, not a guarantee; speed and power are not estimated.

The two cards

	RTX 5090	RTX 4090
VRAM	32 GB	24 GB
Memory bandwidth (speed, not estimated)	1792 GB/s	1008 GB/s

What each runs (~4-bit, max context that fits)

Model	RTX 5090	RTX 4090
Hy3	❌ won't fit · 196/32 GB	❌ won't fit · 196/24 GB
GLM-5.2	❌ won't fit · 484/32 GB	❌ won't fit · 484/24 GB
GLM-4.7-Flash	✅ up to 105K · 21.9/32 GB	✅ up to 19K · 21.9/24 GB
gpt-oss-20b	✅ up to 131K · 15.9/32 GB	✅ up to 131K · 15.9/24 GB
gpt-oss-120b	❌ won't fit · 77.2/32 GB	❌ won't fit · 77.2/24 GB
Qwen 3.6 35B-A3B	✅ up to 117K · 24.8/32 GB	❌ won't fit · 24.8/24 GB
Qwen 3.6 27B	✅ up to 110K · 20.2/32 GB	✅ up to 34K · 20.2/24 GB
Qwen-AgentWorld-35B-A3B	✅ up to 120K · 24.6/32 GB	❌ won't fit · 24.6/24 GB
Gemma 4 31b	✅ up to 67K · 23.5/32 GB	⚠️ up to 3K · 23.5/24 GB
Gemma 4 26b A4B	✅ up to 229K · 18.9/32 GB	✅ up to 83K · 18.9/24 GB
Gemma 4 12b	✅ up to 262K · 10.4/32 GB	✅ up to 262K · 10.4/24 GB
Llama-3.1-8B-Instruct	✅ up to 131K · 8.5/32 GB	✅ up to 92K · 8.5/24 GB
Llama-3.2-3B-Instruct	✅ up to 131K · 5.3/32 GB	✅ up to 123K · 5.3/24 GB
MiniCPM5-1B	✅ up to 131K · 3.2/32 GB	✅ up to 131K · 3.2/24 GB

Only the RTX 5090 runs: Qwen 3.6 35B-A3B, Qwen-AgentWorld-35B-A3B, Gemma 4 31b.

Bottom line

For local LLMs, more VRAM means more models and longer context. Match the card to the model you actually want to run — see the per-model fit pages.

All numbers are computed by the open-source fitllm-engine (MIT) from official model config.json values — reproduce or audit them yourself. Estimates; real usage varies with runtime (llama.cpp / MLX / Ollama), driver and display. Found a mismatch? Report it. · FitLLM home