Upstream contribution · #365
Integrate mlx-audio: STT, TTS, and STS engines
The server had no speech layer. This added three engine classes and changed how models are classified, loaded, and evicted — not just new endpoints.
The server previously understood language models and vision-language models. What this added was not a few endpoints but a new class of engine: all three speech engines use a single forward pass with no KV cache, so their memory policy differs from an LLM's, which meant changing the core path along which models are classified, loaded, and evicted.
The upstream speech library does not present one interface across model families, so one layer of this is a family adapter. mlx-audio is an optional dependency with lazy imports, so deployments that do not use speech are unaffected.
It is the largest of these contributions and the only one that touched the server core. The code is still on main, and other contributors have since built on it.