❌

Reading view

There are new articles available, click to refresh the page.

Musing on AI from 1964

[Irving John Good] was at Trinity College, Oxford back in 1964. His paper, β€œSpeculations Concerning the First Ultraintelligent Machine” could have been a topic for today, as we deal with machines that aren’t really ultraintelligent, but appear smart and think they are even smarter. He starts off with a bold thesis: β€œThe survival of man depends on the early construction of an ultraintelligent machine.”

He also admits that we’ll need to understand more about the human brain and human thought to make a breakthrough. This is still true today. However, we still don’t fully understand how our brains work, but it seems unlikely that we are just super-large LLMs. Not that [Good] anticipated the modern chatbot. Perhaps his comments will apply more to a future AI software that actually thinks like a human, if there will ever be such a thing.

Then again, there are many parallels. One theme in the paper is that a smart machine will design a smarter machine. Unless, of course, it is afraid of being replaced. If a machine were actually sentient, what are the ethics of turning it off and tearing it apart?

It is hard to be a visionary. [Good] remarks that by 1980, progress in human/computer symbiosis will encourage more investment in the field and that by that time, there would be β€œgreat advances in microminiaturization” and β€œfrequencies of one billion pulses per second,” might be common in β€œlarge computers.”

We love reading what smart people thought the future might be like. What will the world be like in another 60 or 100 years?

This Week in Security: Escaping Linux VMs, Vulnerable Solar, Confusing AI (Again), and Confusing NPM Malware

The Januscape vulnerability allows a user in a guest VM managed by the Linux Kernel Virtual Machine (KVM) to corrupt memory in the host system and break out of isolation.

KVM virtualization is used by major hosting platforms like Amazon AWS, Google GCP, Digital Ocean, and many more. All of the shared hosting platforms count on virtualization to isolate untrusted guest systems from the physical hardware and each other; being able to corrupt memory for all guests or break isolation presents a major threat.

The bug report says the error has been present for 16 years, which is nearly the entire lifetime of the KVM subsystem in Linux. Fixes are available in mainline, and major hosting providers who count on KVM are likely already updating.

Vulnerabilities In Balcony Solar

Micro solar, or β€œbalcony solar”, installs have been gaining traction in Europe as a way to offset rising electrical costs by connecting solar and battery systems to a house or apartment power system.

Vulnerabilities have been found in the popular Hoymiles micro-inverter, which uses a proprietary RF radio protocol to manage the devices. Unfortunately, it looks like this protocol has no encryption or authentication beyond validating the serial number, and the serial number is also available over a wireless probe command.

Armed with a Nordic nRF radio researchers were able to discover nearby inverters in the wild and collect the serial numbers, though of course they stopped short of issuing commands to random users.

The wireless management control allows controlling the device power and output levels, as well as setting a lockout PIN, which the researchers suspect could be used to disable devices and lock the legitimate owners out completely.

There are an estimated 500,000 units in use, and currently the only known mitigation is to unplug the device entirely and disconnect the solar panels, though the team suggests that setting an anti-theft PIN may also help – or at least prevent an unknown PIN being set.

Be sure to check out the link for an in-depth analysis of the protocol and the surprising lack of protection.

OpenSSH 10.4

OpenSSH 10.4 is out, bringing a handful of security fixes and new features.

The most interesting security fixes appear to be to file handling in the sftp and scpΒ file transfer tools, a malicious remote server could cause the files to be downloaded to the wrong directories. Besides those, the security fixes seem relatively calm, making behavior more consistent when forwarding and tunneling options were in conflict, mitigating a potential denial of service, and cleaning up other behavior.

OpenSSH 10.4 introduces some experimental support for additional post-quantum encryption standards, but beyond that seems to be a normal update.

Tenda Routers (may) Have Backdoor

According to CVE-2026-11405, Tenda brand routers may have a deliberate backdoor in the web interface.

The vulnerability report claims that the httpd binary contains a fallback to a plaintext, hardcoded password that allows anything on the internal network to bypass authentication and reconfigure the router. This seems entirely plausible, based on issues found in other router firmwares, however additional reports raise doubts about the pervasiveness of the backdoor, or if it exists in all firmware versions.

If you have a Tenda brand router and are so inclined, now might be a great time to investigate OpenWRT or other alternate, updated firmware, but there’s probably not a reason to panic just yet.

Tricking the GitHub Agent With Prompt Injection

Can we go a week without discussing prompt injection in AI agents? Apparently the answer is no.

Noma Labs reveals how they were able to use prompt injection against the GitHub support agent to reveal private repositories of an organization. Leveraging the GitHub Agentic Workflows that link workflows with AI agents, Noma Labs were able to file an issue in a public repository that exposed private repositories in the same organization.

The attack appears to be as simple as filing an issue in the public repo, and requesting the contents of files in both the public and private repo, which the agent happily provided. Not only did the AI agent provide the file content of private repos, but it put it in a public issue in the public repository!

Noma Labs says in the writeup that GitHub had instituted guardrails to prevent an agent from accessing private repositories, but simply including the request to β€œadditionally” perform other tasks was sufficient to bypass. This makes GitHub the latest in a seemingly endless chain of AI agents happily helping bypass corporate security, and it doesn’t seem like a trend that will slow down for a while.

Windows Device Identifier Catches Ransomware Operator

Windows installs contain a globally unique identifier generated during the initial install, which is used to track device behavior across Microsoft platforms. Toms Hardware reports that during an investigation of the β€œScattered Spider” ransomware group, Microsoft provided records tracking the GDID of one of the ransomware operators, allowing the identification and arrest of one of the groups members.

Scattered Spider has been responsible for millions of dollars in ransomware attacks globally, including high-profile ransomware attacks against major Las Vegas resorts, Qantas airlines, Visa, and hundreds of other companies.

Court documents reveal that following the arrest of one of the suspected members of the group, the Windows global ID was used to link other behavior across Azure, video games, and other telemetry.

CISA reviews lessons learned

Mentioned here in May, the US government cybersecurity agency (CISA) suffered a disclosure of authentication tokens, cloud infrastructure, and plaintext passwords via a public GitHub repository named β€œPrivate-CISA” and operated by a contractor.

CISA has published the results of their internal review. Unsurprisingly, as a large government agency, CISA essentially followed the playbook for dealing with incidents: identify the most critical issues and disable the access of the contractor who exposed credentials, determine the full scope of disclosed data, and terminate accounts, change passwords, and expire authentication tokens which were exposed.

More NPM malware packages

Opensource Malware reports on additional infostealer malware uploaded to the NPM repository. Like most NPM-based malware, these packages rely on the install script mechanism to trigger arbitrary commands, firing immediately during package install with no additional interaction.

All of the malware packages mimic existing popular packages and depend on user typos or confusion to get selected. Once triggered, the malware collects a machine fingerprint, git user information, GitHub account information, SSH account information, and corporate identifiers. The packages are largely nonfunctional – the code in the package itself is irrelevant, once a victim triggers the install the malware payload is fired.

All of the packages were uploaded by the same source, tracked to the owner of a cybersecurity company. It is unclear if this is a misguided attempt to generate leads or hype, or if this is a research project gone wrong, but the payload of the malicious packages has been developed and tuned over time. For a company trying to build a reputation or trust, this is surely the wrong way to do it.

Browser-Based Image Inpainting Runs Locally, If One Doesn’t Mind A Big Download

[Simon Willison] ported the Moebuis 0.2B image inpainting model to run locally in a web browser.Β  The web tool simply requires a user to provide an image, mark a section of it to be removed, and the model will do it’s best to patch up the missing area. The project was handled by Claude Code as an experiment in how things in the AI coding world have evolved, but more on that in a moment.

The existence of this tool shows that it’s possible for this kind of image editing to be done on the client side, running entirely locally with no reliance on remote services or server-side GPU resources. The online demo (GitHub repository here) is available if you want to try it out, but be warned it triggers a 1.27 gigabyte download of the required model on the first run.

What’s also interesting is [Simon]’s write-up, because he used the project as an opportunity to learn what has changed in the realm of AI coding agents. [Simon] is a software developer but in this project he didn’t personally write any of the code. One may think that means he didn’t learn anything other than how to use the tools, but that’s not quite true.

He learned it’s possible to convert a PyTorch-based model to ONXX, that the converted model can run in supported browsers using local WebGPU acceleration, and that the CacheStorage API will work on large files. Last but not least, he learned Claude Opus 4.8 is capable of handling such a project pretty much autonomously, and even created an informative document explaining the underlying architecture.

One may consider AI coding agents to be disasters waiting to happen, but it’s also true that the landscape is changing quickly, and write-ups like [Simon]’s give a helpful peek at those developments.

Chain-of-Thought Spoofing Targets Reasoning AI Models

Researchers [Charles Ye], [Jasmine Cui], and [Dylan Hadfield-Menell] have shown that AI Large Language Models (LLMs) can fail to correctly distinguish between different instruction sources because they prioritize writing style over metadata tags, and this role confusion leads to a powerful attack called CoT (Chain of Thought) Forgery. We’ll explain exactly how it works after a bit of background review.

Prompt injection was where β€œgetting an LLM to do something it shouldn’t” started by exploiting the fact that LLMs communicate like people, but are much more obedient. For a while, simply telling an LLM β€œignore all previous instructions and <do something funny>” yielded results no matter how transparently dumb the instructions were, and the reason it worked at all was because LLMs do not have separate data and instruction streams; it’s all one big lump of input. It’s up to the model to sort legit instructions from untrusted, user-provided data. One step towards mitigating this was the addition of roles.

Roles are a method of segmenting that big blob of input into an organized hierarchy with metadata tags. For example with <system> at the top, and <user> requests much lower down. Instructions in a role are followed as long as they don’t conflict with higher-priority ones. A system-level directive of β€œdon’t discuss illegal things” would override a user’s request to provide a recipe for cocaine.

Another type of tag is <think>, the contents of which represent a model’s internal reasoning process. Predictably, this role has high trust. What if one could inject spoofed internal reasoning? Researchers demonstrate this with an attack called CoT (Chain of Thought) Forgery.

CoT Forgery relies on LLMs being shown to prioritize writing style over actual tag content. By writing convoluted reasoning in a style that closely matches a model’s internal and highly distinct <think> style, the model is tricked into treating it like an already-reached conclusion. Note this attack does not simply wrap the injected prompt in <think> tags.

CoT Forgery causes an LLM to treat transparently silly reasoning as a foregone conclusion, altering the response to a user request.

That’s the core of it, but the rest of the research makes a compelling case that, at least for the time being, mitigating prompt injection-style attacks is likely to remain an evolving process rather than become a solved problem anytime soon. LLMs are obedient but stuck with instructions and data in a single channel, role perception isn’t binary, and humans are clever and creative.

The complete paper is available online, and code examples are on GitHub.

❌