The Bottleneck Is Memory: Notes From a Weekend Local AI Server

Local AI is already useful for private and bulk tasks, but memory—not processing power—is the bottleneck that limits both consumer systems and the AI buildout.

• A repurposed RTX 3090 runs a 27-billion-parameter model that can write faster than the author reads, process PDFs, write code and hold about 300 pages, while keeping private documents on the premises.
• Smaller models trained on financial text can match much larger models on sentiment analysis and headline classification in published benchmarks, but still trail frontier models on difficult reasoning; the proposed rule is local models for private or bulk work and frontier subscriptions for judgment.
• Adding a second graphics card increased available memory rather than speed, because language models need the full model for each generated word; the 27-billion-parameter model requires about 54 GB at full precision, versus the card's 24 GB, and about 14 GB after reduced-precision rounding.
• Frontier models with a trillion or more parameters would require roughly half a terabyte of similarly rounded memory, equivalent to about twenty consumer cards or specialized accelerators using high-bandwidth memory.
• Memory prices paid by bulk buyers more than doubled year over year, Micron said AI accelerator memory uses about three times as much factory capacity as ordinary PC memory, and more than one-fifth of 2026 memory-chip production is directed to AI accelerators; the author's share-price baskets reached mid-September gains of about 19-fold for storage, 16-fold for memory and eight-fold for processors from 30 November 2022.

Takeaway: The AI buildout is repricing memory, storage and power rather than simply delivering cheaper hardware to consumers; memory scarcity supports accelerator demand through the current capacity expansion, but the market has already priced in much of that scarcity, consumer conditions are flattening, and the article presents this as a structural framework rather than a forecast.