Wyoming Onnx ASR

Wyoming protocol server for the onnx-asr speech to text system.

Docker Image

docker run -it -p 10300:10300 -v /path/to/local/data:/data ghcr.io/tboby/wyoming-onnx-asr

or for gpu

docker run -it -p 10300:10300 --gpus all -v /path/to/local/data:/data ghcr.io/tboby/wyoming-onnx-asr-gpu

There is also gpu TensorRT support, but it's a huge image and doesn't seem to make much performance difference. You might want to mount in a cache folder if using it (/cache).

Local Install

Install uv

Clone the repository and use uv:

git clone https://github.com/tboby/wyoming-onnx-asr.git
cd wyoming-onnx-asr
uv sync

Run a server anyone can connect to:

uv run --uri 'tcp://0.0.0.0:10300'

The --model-en or --model-multilingual can also be a HuggingFace model but see onnx-asr for details

NOTE: Models are downloaded temporarily to the HF_HUB_CACHE directory, which defaults to ~/.cache/huggingface/hub. You may need to adjust this environment variable when using a read-only root filesystem (e.g., HF_HUB_CACHE=/tmp).

Configuration

• Quantization: the parakeet model supports int8, but make sure to compare as performance may or may not improve.

Running tooling

Install mise and use mise run to get a list of tasks to test, format, lint, run.