Login
Linux Fu: Speak Up!
Apparently, people hate typing. As every movie and TV show suggests, the future is talking to computers. There was a time when speech recognition was complex and not very good. But these days, even our lowly phones can do a pretty good job of speech recognition. Of course, one problem is that your phone probably isn’t actually doing the speech recognition. It sends it to the big business of your choice to interpret. I’ve been using Handy, a speech recognition system that works well for me. I’ve also looked at some that didn’t.
After all, it is sometimes nice to dictate to your computer, and it would be even nicer if you could keep your data local. On Windows, oddly enough, there is a well-developed speech feature that, as far as I can tell, almost no one talks about or uses. One video estimates that 99% of users don’t use it. Linux, of course, has many options, but historically, these have been difficult to set up or finicky.
Of course, the good news is that many of the Linux tools are open source and the models are quite good. That means other people have had the freedom to fork the tools and make them easier to use, at least in theory. The licensing of the models themselves may be different, but those will be hard to modify, anyway and they generally work well. The biggest problems on Linux isn’t the technology itself, but the tremendous variety of systems and setups.
Suppose you want to write a speech-to-text program. Will it work on ARM? What desktops will it integrate with? Can it use a GPU? What kind? What about specialized instructions in some CPUs? Then there’s the forced input situation; typing into arbitrary programs once you know what the user said. On X11, it is easy, but Wayland needs different handling.
A Shortcut
I’ve thought about using my phone with KDE Connect, which is an excellent program. It can let you use your phone as a keyboard and mouse for your Linux computer. Unfortunately, it is aimed at character-at-a-time input, and I’ve never found a way to make it work with voice.
Besides, the phone is beaming all the data to “the cloud.” You probably type things you’d rather not broadcast to the ether.
I had looked at Speech Note before, but it is sort of a speech recognition notepad. I didn’t find it seamless, and it didn’t work well on my system anyway. Vocalinux looks nice, but a quick test kept complaining that my Intel extensions were not available. Makes sense, since I have an AMD CPU. Even though the documentation said it should work, I was never able to get it to work.
The Easy Way
Turns out the application that worked readily on my machine was Handy. Keep in mind, Handy is just another tool that uses one of several models out there, along with other open-source tools. You might need to install some tools to deal with your system like xdotool or dotool, but they are probably already installed anyway. That isn’t to minimize the value of Handy. It is — well — Handy. You don’t have to load and configure models, set up a bunch of system-level hooks, or install a bunch of libraries. You install it, and it works.
You can configure it. The best model for you, for example, may depend on your machine and the languages you speak. You can configure the hotkeys and how the app types into your computer. But it does all the work of downloading and configuration.
No Cloud, Unless…
The models do run on your computer and you can make sure it takes advantage of your hardware. However, there is an optional alternate hotkey that takes your speech, processes it to text, and then sends it to your choice of AI engines to clean it up.
Of course, you could be running your own AI engine, but normally you’ll have it sent somewhere else with a prompt. You can tune the prompt or create your own, but the default one starts: “Clean this transcript: 1. Fix spelling, capitalization, and punctuation errors 2. Convert number words to digits (twenty-five → 25, ten percent → 10%, five dollars → $5) 3. Replace spoken punctuation with symbols (period → ., comma → ,, question mark → ?) 4. Remove filler words (um, uh, like as filler)…”
You do need an API key, but there are free options available. For experimenting purposes, I went to OpenRouter, generated a key, and attached it to one of several free models they have. The nice thing is that you can experiment with different models while keeping the same key.
If you search for free in the models box, you will find a few choices including openrouter/free which just picks a free model that isn’t too busy. That can be important because some of the models will introduce long wait times into your transcription.
On the other hand, you can make a new prompt, copy the original one in, delete the part about keeping the language the same, and add instructions to translate the output to French, and that will work, at least most of the time. So there are a lot of possibilities.
Rather than tell you all about it, we’d encourage you to install it and try it or watch the reveiw video below.
Special Mention
Although Handy is my first choice for day-to-day transcription use, there is another open source project that’s worth mentioning. Nerd Dictation is a very lightweight wrapper around the Vosk model. It does take a little bit to set up, and then it provides you with a command line tool that can start and stop dictation. Of course, you can assign those to macro keys. However, there is also a switch that allows you to simply output to stdout. That opens up a lot of possibilities for writing programs or even shell scripts that respond to voice.
To see what’s possible, run nerd-dictation begin --help. This will show you how to output to stdout, set a timeout, and handle other options.
Of course, the obvious project would be a voice typewriter. Many of the tools mentioned here either rely on or can use OpenWhisper and, of course, you can use it too, if you roll your own code.
