LLMs: Work Smarter, Not Harder. Or — 16GB Is Plenty
This is one of those "Armen discovers electricity" kind of posts. Feel free to skip it, especially if you're experienced in the area of Large language models and/or are fed up with seeing articles on LLMs and "AI" popping up in every corner of the internet for the past five or so years.
I'm a hardcore "optimization guy". If there's something that can be made smaller/faster/more efficient — I'll be on it like a dog with a bone.
My first GPU was NVIDIA GeForce2 MX with only 32 Megabytes of memory. It ran almost anything like a champ in 2001/2002: Quake III Arena, Unreal Tournament, Half-Life & Counter-Strike, Deus Ex, Max Payne and, of course, GTA III. But as it got older, it started showing its age by struggling to keep up with The Elder Scrolls III: Morrowind which I was really into.
This set me on quite an upgrade journey over the following 25 years:
- GeForce4 Ti 4200
- GeForce FX 5200
- GeForce 6600 GT
- GeForce 8600 GT
- GeForce GTX 275
- GeForce GTX 580
- GeForce GTX 970
- GeForce GTX 1070 + GeForce GTX 1080 Ti
And, finally — my amazing, state-of-the-art 16GB GeForce RTX 4080 ❤️
There's something wonderful about going from 32MB of VRAM to 16GB. That's a 512-fold increase in graphics memory!
I gradually moved away from gaming and started using my computer more for creative and professional tasks like video editing, special effects, 3D, and in the last couple of years — image and text generation/manipulation with Large language models.
Recently I made a curious observation: when it comes to LLMs, we're getting better at using hardware that already exists, rather than waiting for GPUs with several hundred gigabytes of VRAM to become affordable. I can clearly see this when running local Large Language Models (like Gemma 4 or QWEN3.8) — over time, token generation speeds have been steadily increasing, seemingly without any extra effort (or new hardware) required from me.