I send my assistant voice notes in Telegram when it’s easier to talk through an idea than type it. The bot retrieves the recording from Telegram, and whisper.cpp runs the speech-to-text step on my hardware. Joanna receives the words and can follow up with Home Assistant when a request calls for it.
Where Whisper runs in my setup
Proxmox hosts an Ubuntu Linux virtual machine. Docker runs in the VM, and the container uses FFmpeg to convert the voice note to 16 kHz mono WAV. The English base.en model runs on the CPU. In the video I use the original JFK sample and a staged garage question to check the output directly.
Try a transcription on your machine
Follow the whisper.cpp build and model instructions to install whisper-cli and download the model. Put the model where the executable can reach it. Then run this from the directory with a sample Telegram voice file, adjusting the model path to match your install.
ffmpeg -i message.ogg -ar 16000 -ac 1 -c:a pcm_s16le message.wav
whisper-cli -m models/ggml-base.en.bin -f message.wav -l en -nt -otxt -of message
Read message.txt before connecting the assistant. Try a recording where you change your mind halfway through. I asked about the light first, then asked whether the garage door was closed. I wanted to see both questions in the transcript so Joanna would have the correction. The upstream JFK recording gives you a second input to try.
Wire the transcript into the bot
- Queue the Telegram voice attachment so the bot can acknowledge the request promptly.
- Retrieve the file with the Telegram Bot API and convert it locally. Delete the temporary files after transcription.
- Pass the text to the same handler you use for typed requests. Let your assistant decide if it needs a Home Assistant lookup, using the permissions you give it.
Telegram still carries the recording. On my setup, model inference runs in Docker on the Linux VM. The assistant’s tools and later processing follow their own configuration.
If the transcript comes back empty
Play the WAV file to check that the audio survived conversion. Verify the model path and, if Docker runs the executable, check that the model and WAV are visible inside the container. The sample uses the English-only model. For other languages, start with a multilingual model and the corresponding language setting.
I use the same idea with cameras. Frigate gives Home Assistant a state it can use for package alerts and reminders about the bins. Giving the model one clear job makes the result easier to check before the automation uses it.
Watch the local transcription demo
Watch I Run Whisper Locally for My Telegram Voice Notes on YouTube