Upstream contribution · #365

Integrate mlx-audio: STT, TTS, and STS engines

The server had no speech layer. This added three engine classes and changed how models are classified, loaded, and evicted — not just new endpoints.

Status
Merged
Opened
Merged
Diff
+3413 / −38
Area
Engines and model integration

The server previously understood language models and vision-language models. What this added was not a few endpoints but a new class of engine: all three speech engines use a single forward pass with no KV cache, so their memory policy differs from an LLM's, which meant changing the core path along which models are classified, loaded, and evicted.

The upstream speech library does not present one interface across model families, so one layer of this is a family adapter. mlx-audio is an optional dependency with lazy imports, so deployments that do not use speech are unaffected.

It is the largest of these contributions and the only one that touched the server core. The code is still on main, and other contributors have since built on it.