In the business world, a moat is a quality of a business that makes it difficult for competitors to take that company’s profits. With how hard it is to train models for large language models (LLMs) and generative AI, it might seem like Anthropic, Open AI, and other LLM companies would have huge moats given the amount of compute it takes to build models. But open source models are quickly draining that moat, and now the only thing standing in the way of a customer using one of these models on their own hardware instead one from the larger companies is physical computing resources. [TerminalBytes] demonstrates a few of these models on personally owned computers to show the current state of the art.

[TerminalBytes] started off running the 27B version of the Qwen3.8 on a Mac Studio with 256 GB of unified RAM, which is plenty for this task. But it’s also enough to benchmark a few different models. Qwen3.6 is compared to 3.8, and then the different quants of each model are also compared. Quants are compressed versions of models that need fewer bits to store weights, meaning that the same models can run in less memory with smaller losses in fidelity. Many of these quants run on machines with 32 GB of RAM or less, encompassing many average gaming PCs. There’s even a 1-bit quant that [TerminalBytes] tested which can easily run on a machine with 16 GB, although with mixed results.

Keep in mind that this is just the current state of affairs with open LLMs. Future versions of these models are likely to optimize the number of tokens produced per unit time, or otherwise increase quality of responses while requiring less computer resources. We don’t really think that the ease of running local models will be the sole reason that the AI bubble pops, though. The fact that not every computer user is running Linux is proof enough of that.

We all know that the price of RAM and storage has shot up due to demand from the AI industry and other factors. In most cases that means we grumble about the price, but if we really need the part we can fork out for it. [CNX Software] are reporting that rather than continue to push up their prices, Pine64 are responding to the crisis by halting production of their Linux boards for the time being.

We’ve seen online comment over the now-exorbitant cost of other boards such as a fully-loaded Raspberry Pi, and this follows in that vein. If we had to guess we’d speculate that the high process have resulted in too little in the way of sales, which considering the knock-on impact on our community if other vendors follow suit, could be concerning.

If there’s one bright spot in the current situation, it’s that for many applications where a single-board computer might be used, a microcontroller might now be a better choice for the job than something running Linux. We’re in a very different situation from that we were in when cheap Linux boards appeared, the current generation of high-power microcontrollers have significantly closed the gap between the two. Given that microcontrollers have onboard memory and storage, their immunity from the price hikes makes them even more attractive. As to Pine64, we hope that sales of their other products make up for it.

Header image: BasilicumTree, CC0.