Run Whisper Locally for Telegram Voice Notes

Follow a Telegram voice note through local whisper.cpp transcription in Docker, then hand the text to a Home Assistant-capable assistant.

Watch on YouTube ↗Explore the gear →

I send my assistant voice notes in Telegram when it’s easier to talk through an idea than type it. The bot retrieves the recording from Telegram, and whisper.cpp runs the speech-to-text step on my hardware. Joanna receives the words and can follow up with Home Assistant when a request calls for it.

LOCAL VOICE TO TEXT and a phone showing an audio waveform becoming text

Where Whisper runs in my setup

Proxmox hosts an Ubuntu Linux virtual machine. Docker runs in the VM, and the container uses FFmpeg to convert the voice note to 16 kHz mono WAV. The English base.en model runs on the CPU. In the video I use the original JFK sample and a staged garage question to check the output directly.

Try a transcription on your machine

Follow the whisper.cpp build and model instructions to install whisper-cli and download the model. Put the model where the executable can reach it. Then run this from the directory with a sample Telegram voice file, adjusting the model path to match your install.

ffmpeg -i message.ogg -ar 16000 -ac 1 -c:a pcm_s16le message.wav
whisper-cli -m models/ggml-base.en.bin -f message.wav -l en -nt -otxt -of message

Read message.txt before connecting the assistant. Try a recording where you change your mind halfway through. I asked about the light first, then asked whether the garage door was closed. I wanted to see both questions in the transcript so Joanna would have the correction. The upstream JFK recording gives you a second input to try.

Wire the transcript into the bot

  • Queue the Telegram voice attachment so the bot can acknowledge the request promptly.
  • Retrieve the file with the Telegram Bot API and convert it locally. Delete the temporary files after transcription.
  • Pass the text to the same handler you use for typed requests. Let your assistant decide if it needs a Home Assistant lookup, using the permissions you give it.

Telegram still carries the recording. On my setup, model inference runs in Docker on the Linux VM. The assistant’s tools and later processing follow their own configuration.

If the transcript comes back empty

Play the WAV file to check that the audio survived conversion. Verify the model path and, if Docker runs the executable, check that the model and WAV are visible inside the container. The sample uses the English-only model. For other languages, start with a multilingual model and the corresponding language setting.

I use the same idea with cameras. Frigate gives Home Assistant a state it can use for package alerts and reminders about the bins. Giving the model one clear job makes the result easier to check before the automation uses it.

Watch the local transcription demo

Watch I Run Whisper Locally for My Telegram Voice Notes on YouTube

Advertisement

Keep building

The next useful idea, in your inbox.

Get new vCloudInfo posts by email.

Get new posts