Offline voice loop on your PC with Ollama and LM Studio

You are going to wire three pieces: the model, speech to text, and text to speech. Ollama runs the LLM on CPU. LM Studio gives you a simple chat window if you want a GUI. For speech, use a tiny Python script with faster-whisper (a CPU speech-to-text engine) and your system’s built-in voice.

Start by installing Ollama and pulling a small CPU model. On macOS or Linux, run:

bash
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3b

Verify the model runs locally:

bash
ollama run llama3.2:3b "Write one sentence about local AI."

If you prefer a GUI, install LM Studio (a desktop app for running and chatting with local models). Keep Ollama running for the API. Next, you will add offline voice. Install Python 3.10+, ffmpeg (command line audio tool), and a small script that records 5 seconds of audio, transcribes it locally, sends it to Ollama, then speaks the reply. That is the loop. The full guide has per platform setup, the script, and tweaks.

You are going to wire three pieces: the model, speech to text, and text to speech. Ollama runs the LLM on CPU. LM Studio gives you a simple chat window if you want a GUI. For speech, use a tiny Python script with faster-whisper (a CPU speech-to-text engine) and your system’s built-in voice.

Start by installing Ollama and pulling a small CPU model. On macOS or Linux, run:

bash
curl -fsSL https://ollama.com/install.sh | sh
ollama pull llama3.2:3b

Verify the model runs locally:

bash
ollama run llama3.2:3b "Write one sentence about local AI."

If you prefer a GUI, install LM Studio (a desktop app for running and chatting with local models). Keep Ollama running for the API. Next, you will add offline voice. Install Python 3.10+, ffmpeg (command line audio tool), and a small script that records 5 seconds of audio, transcribes it locally, sends it to Ollama, then speaks the reply. That is the loop. The full guide has per platform setup, the script, and tweaks.

Setup

macOS

• Ollama (local LLM runner):

bash
  brew install --cask ollama
  ollama pull llama3.2:3b
  

• LM Studio (desktop chat UI for local models):

bash
  brew install --cask lm-studio
  

• ffmpeg (command line audio tool):

bash
  brew install ffmpeg
  

• Python bits for offline STT and the loop:

bash
  python3 -m pip install --upgrade pip
  python3 -m pip install faster-whisper requests sounddevice soundfile
  

Note: macOS already has TTS via the say command. No extra TTS install needed.

Linux (Debian/Ubuntu)

• Ollama:

bash
  curl -fsSL https://ollama.com/install.sh | sh
  sudo systemctl enable ollama
  sudo systemctl start ollama
  ollama pull llama3.2:3b
  

• ffmpeg and TTS:

bash
  sudo apt update
  sudo apt install -y ffmpeg espeak-ng python3-pip python3-venv
  

• Python bits:

bash
  python3 -m pip install --upgrade pip
  python3 -m pip install faster-whisper requests sounddevice soundfile
  

Windows 11

• Ollama:

bash
  winget install Ollama.Ollama
  ollama pull llama3.2:3b
  

• ffmpeg:

bash
  winget install Gyan.FFmpeg
  

• Python bits:

1. Install Python 3.10+ from Microsoft Store or python.org, then in PowerShell:

bash
     py -m pip install --upgrade pip
     py -m pip install faster-whisper requests sounddevice soundfile
     

- Windows TTS uses built in SAPI via PowerShell in the script.

Create the voice loop script

Create local_voice_loop.py in a working folder.

py
import os
import sys
import json
import time
import platform
import subprocess
import requests
import numpy as np
import sounddevice as sd
import soundfile as sf
from faster_whisper import WhisperModel

MODEL_NAME = os.environ.get("OLLAMA_MODEL", "llama3.2:3b")
OLLAMA_URL = os.environ.get("OLLAMA_URL", "http://127.0.0.1:11434/api/generate")
REC_SECONDS = int(os.environ.get("REC_SECONDS", "5"))
SAMPLE_RATE = 16000
WAV_PATH = "recording.wav"

print(f"Model: {MODEL_NAME}")

# load STT model once (tiny.en is fine for quick tests on CPU)
whisper = WhisperModel("tiny.en", device="cpu")

def record_wav(seconds=REC_SECONDS, sr=SAMPLE_RATE):
    print(f"Speak for {seconds}s...")
    audio = sd.rec(int(seconds * sr), samplerate=sr, channels=1, dtype='float32')
    sd.wait()
    sf.write(WAV_PATH, audio, sr)
    return WAV_PATH

def transcribe(path):
    segments, info = whisper.transcribe(path, vad_filter=True)
    text = "".join([seg.text for seg in segments]).strip()
    return text

def ask_ollama(prompt):
    payload = {"model": MODEL_NAME, "prompt": prompt, "stream": False}
    r = requests.post(OLLAMA_URL, json=payload, timeout=600)
    r.raise_for_status()
    data = r.json()
    return data.get("response", "")

def speak(text):
    sys_os = platform.system()
    if sys_os == "Darwin":
        subprocess.run(["say", text])
    elif sys_os == "Linux":
        subprocess.run(["espeak-ng", "-s", "165", text])
    elif sys_os == "Windows":
        ps = f"Add-Type -AssemblyName System.Speech;($s=new-object System.Speech.Synthesis.SpeechSynthesizer).Speak(\"{text.replace('\\', ' ').replace('"', '\'')}\")"
        subprocess.run(["powershell", "-NoProfile", "-Command", ps])
    else:
        print("TTS not supported on this OS. Text:", text)

print("Press Enter to record, or type q then Enter to quit.")
while True:
    cmd = input("> ").strip().lower()
    if cmd == 'q':
        break
    path = record_wav()
    prompt = transcribe(path)
    if not prompt:
        print("Heard nothing. Try again.")
        continue
    print(f"You said: {prompt}")
    reply = ask_ollama(prompt)
    print(f"AI: {reply}")
    speak(reply)

Tip: set the model with an env var if you want a smaller or larger one, for example export OLLAMA_MODEL=qwen2.5:1.5b (Qwen2.5 is a small CPU friendly model in Ollama).

Verify

• Ollama is up and serving a model:

bash
  ollama --version
  # ollama version 0.3.14

  ollama run llama3.2:3b "Say 'ready'."
  # ready
  

• API replies:

bash
  curl -s http://127.0.0.1:11434/api/tags | grep llama3.2
  # "name":"llama3.2:3b"
  

• The loop script runs and speaks:

bash
  python3 local_voice_loop.py
  # Model: llama3.2:3b
  # Press Enter to record, or type q then Enter to quit.
  # >
  # Speak for 5s...
  # You said: summarize this note into one sentence
  # AI: A one sentence summary...
  # (voice plays)
  

Configuration tips

• Pick a smaller model for CPU. Examples in Ollama: qwen2.5:1.5b, phi3:mini, llama3.2:1b. Pull then set export OLLAMA_MODEL=qwen2.5:1.5b.
• Lower generation cost for snappy replies:

bash
  ollama run $OLLAMA_MODEL -p "You are concise. Answer in one sentence."
  

Or use the API with { "options": { "num_ctx": 2048, "temperature": 0.2, "top_p": 0.9 } }.

• Tune faster-whisper size if CPU is slow. Use WhisperModel("tiny.en") for speed, base.en for accuracy, small.en if you have time.
• Change TTS voice:

- macOS: say -v Samantha "text" - Linux: espeak-ng -v en-us+m3 "text" - Windows: set default system voice in Settings. The script will use it.

• Hand off to your workflow. Replace the speak(reply) line with a call that writes to a notes file, appends to a CRM CSV, or triggers a local script.

Troubleshooting

• Port in use on 11434:

- Symptom: requests.exceptions.ConnectionError to Ollama. - Fix: pgrep ollama then kill it, or change OLLAMA_URL to a free port and start ollama serve -p 11435.

• faster-whisper tries to download models and fails offline:

- Symptom: long hang on first run with network error. - Fix: run the script once while online to cache the tiny model. After that, it runs offline. Cache lives under ~/.cache/ctranslate2.

• sounddevice device error:

- Symptom: PortAudioError: Error opening InputStream. - Fix: pick a valid input device.

py
    import sounddevice as sd; print(sd.query_devices())
    # then set sd.default.device = (input_index, None)
    

On Linux, confirm your user is in the audio group and the mic is not in use.

• espeak-ng not found on Linux:

- Install: sudo apt install espeak-ng then rerun.

• Script prints the reply but no voice on Windows:

- Run PowerShell as a normal user, not as admin. Ensure system audio is not muted. Test TTS: powershell -c "Add-Type -AssemblyName System.Speech; (New-Object System.Speech.Synthesis.SpeechSynthesizer).Speak('test')".

When it beats paid cloud chatbots

• You record customer calls or field notes with PII. The audio and text never leave the laptop.
• You work on spotty Wi Fi. The loop still records, transcribes, and drafts a reply.
• You run demos in rooms with no internet. The voice loop keeps working on CPU.
• You need predictable cost. No per minute billing for audio or tokens during tests.

Sources