As technologies change and adapt, we’re often left with seemingly useless junk that has nowhere to go. Certainly anyone still sitting on a pile of floppy disks feels this way sometimes, but odds are anyone who owns a mining ASIC or an NFT can attest to that as well. The trillions of dollars flowing into GPU-based data centers will likely become the next victim of this trend, so if you want to capitalize on the losses of some venture capitalist you’ll want to figure out a way to get GPUs meant for a server into your desktop doing useful work.

Of course, calling these devices GPUs is a bit of a stretch compared to the Radeon and GeForce cards many of us are used to using for gaming. These don’t even have a PCIe slot or video output, after all. But, as [Oscar] notes, the VRAM and GPU cores are very real and can still do useful work. An adapter board is able to mate a Tesla V100 SXM2 16 GB GPU to a standard PCIe slot, which solves the first problem, but the major downside from there is that the cooling fan for this unit was literally deafeningly loud. At 82 dB it was about as loud as a lawnmower, which is fine in a server rack but not great in a bedroom. [Oscar] found a way to tamp down the fan speed, making it usable in a home.

Without video output, the utility of these cards mainly comes from adding VRAM and compute for tasks that benefit from parallel computing. Using tensor splitting, [Oscar] is running a local LLM with this card alongside his RTX 4080, providing 32 GB of VRAM on his NixOS system. With his benchmarking tests, the LLM sports impressive stats for a self-hosted model, ranking somewhere around Claude Sonnet 4.6. What’s even more impressive is that this is all done for around £200, and with the rate the various LLM companies are ratcheting up pricing could pay itself back very quickly. If trading off performance for cost is acceptable, though, it’s possible to run local models on much less powerful hardware as well.

Researchers [Charles Ye], [Jasmine Cui], and [Dylan Hadfield-Menell] have shown that AI Large Language Models (LLMs) can fail to correctly distinguish between different instruction sources because they prioritize writing style over metadata tags, and this role confusion leads to a powerful attack called CoT (Chain of Thought) Forgery. We’ll explain exactly how it works after a bit of background review.

Prompt injection was where “getting an LLM to do something it shouldn’t” started by exploiting the fact that LLMs communicate like people, but are much more obedient. For a while, simply telling an LLM “ignore all previous instructions and <do something funny>” yielded results no matter how transparently dumb the instructions were, and the reason it worked at all was because LLMs do not have separate data and instruction streams; it’s all one big lump of input. It’s up to the model to sort legit instructions from untrusted, user-provided data. One step towards mitigating this was the addition of roles.

Roles are a method of segmenting that big blob of input into an organized hierarchy with metadata tags. For example with <system> at the top, and <user> requests much lower down. Instructions in a role are followed as long as they don’t conflict with higher-priority ones. A system-level directive of “don’t discuss illegal things” would override a user’s request to provide a recipe for cocaine.

Another type of tag is <think>, the contents of which represent a model’s internal reasoning process. Predictably, this role has high trust. What if one could inject spoofed internal reasoning? Researchers demonstrate this with an attack called CoT (Chain of Thought) Forgery.

CoT Forgery relies on LLMs being shown to prioritize writing style over actual tag content. By writing convoluted reasoning in a style that closely matches a model’s internal and highly distinct <think> style, the model is tricked into treating it like an already-reached conclusion. Note this attack does not simply wrap the injected prompt in <think> tags.

That’s the core of it, but the rest of the research makes a compelling case that, at least for the time being, mitigating prompt injection-style attacks is likely to remain an evolving process rather than become a solved problem anytime soon. LLMs are obedient but stuck with instructions and data in a single channel, role perception isn’t binary, and humans are clever and creative.

The complete paper is available online, and code examples are on GitHub.