Project
oMLX upstream contributions
Added a whole class of engines to an open-source LLM inference server for Apple Silicon, then moved into model discovery, API conformance, and the scheduler core.
Abstract
oMLX belongs to someone else; what is recorded here is the verifiable contribution: 13 pull requests, 10 merged, 1 not adopted, and 2 still open and unmerged as of July 27, 2026. Every row below links upstream, and GitHub is authoritative for live status.
oMLX is an open-source LLM inference server for Apple Silicon with continuous batching and SSD caching, managed from the macOS menu bar; Apache-2.0, Python, roughly 18.2k stars when this record was updated. Pull-request statuses in this record were checked through July 27, 2026; follow the GitHub links for live status after that date.
The project is not mine. This record documents contributions and leans on pull request numbers — every row below can be opened upstream and checked, including the one that was not adopted. The largest change integrated the speech engines and in doing so touched the core path that classifies and loads models; the two most recent have left audio for the scheduler.
A few boundaries. An open, unmerged pull request is not a delivery — it has only been submitted. Commit counts are not used as a measure, because some commits carry a git identity not linked to the account and the number is not clean. No usage is claimed: the repository's star count belongs to the project, not to this work. What is checkable is whether code merged, whether it shipped in a release, and whether it is still on main.
Pull requests
Every row links to the upstream pull request. The status below is a static snapshot checked on ; GitHub remains authoritative for the current state, including the change that was not adopted.
13 pull requests · 10 merged · 2 still open · 1 not adoptedMerged: +5,497 / −68
Engines and model integration
| PR | What it did | Status |
|---|---|---|
| #365 | Integrate mlx-audio: STT, TTS, and STS enginesThe server had no speech layer. This added three engine classes and changed how models are classified, loaded, and evicted — not just new endpoints.Notes | Merged |
OpenAI API conformance
| PR | What it did | Status |
|---|---|---|
| #823 | Honor response_format on /v1/audio/speechNot adopted. Three months later the maintainer solved the same problem differently, avoiding an ffmpeg dependency.Notes | Not adopted |
| #2075 | SSE streaming for the transcription endpointThe stream parameter was accepted but ignored. Models without native streaming fall back to one-shot inference and still emit a valid event stream. | Merged |
| #2082 | Support the OpenAI prompt field for STT biasingMaps prompt onto each backend's own biasing hook, and drops it explicitly where no hook exists rather than pretending it applied. | Merged |
| #2084 | Add GET /v1/audio/voices to list speakersAnswered from static config files without loading a model. An oMLX extension, not part of the OpenAI spec. | Merged |
Voice and speaker handling
| PR | What it did | Status |
|---|---|---|
| #467 | Fix voice parameter routing for CustomVoice modelsA parameter-priority bug: instruct was checked before voice, so models accepting both misrouted the speaker name and caused a 500 error. | Merged |
| #676 | Zero-shot voice cloning on /v1/audio/speechAccepts base64 only, never a URL. That is a security tradeoff; the reason is in the notes.Notes | Merged |
Model discovery and diagnosability
| PR | What it did | Status |
|---|---|---|
| #525 | Resolve model aliases in the audio endpointsAliases that worked on the LLM routes returned 404 here. All three endpoints now share one resolver. | Merged |
| #826 | Clear errors for non-LLM engines and missing STT processorsHalf of this is not on an audio path: the chat-completions endpoint accepted any engine, so calling it with a speech model crashed. | Merged |
| #2086 | Fix Kokoro model discovery and G2P inferenceConverted models lacked model_type and were misread as LLMs, producing a misleading "no safetensors" error. Language is now inferred from the voice-name prefix using a deliberately narrow rule. | Merged |
Build and dependencies
| PR | What it did | Status |
|---|---|---|
| #2085 | Cap transformers to unbreak CIAn upstream tightening broke the import, and with it all test collection on fresh environments. Repository-wide, unrelated to audio. | Merged |
Scheduling and speculative prefill
| PR | What it did | Status |
|---|---|---|
| #2346 | Memoize SpecPrefill scoring by prompt hashByte-identical prompts were re-scored every time. The input was cached (draft KV); the output — the selected tokens — was not.Notes | Open, not merged |
| #2361 | Keep external VLM MTP reachable after chunked prefillShort prompts routed correctly; long ones silently fell back with no warning. The routing decision sat on only one exit path.Notes | Open, not merged |