Upstream contribution · #2361
Keep external VLM MTP reachable after chunked prefill
Short prompts routed correctly; long ones silently fell back with no warning. The routing decision sat on only one exit path.
The symptom was asymmetric: short prompts routed onto the MTP fast path correctly, while prompts long enough to trigger chunked prefill fell back to ordinary decoding with no warning at all. Turning chunked prefill off made the same long prompt work again.
The cause is where the routing decision physically sits. Exactly one place in the codebase decides whether a request takes that path, and it sits on the non-chunked exit; chunked requests leave earlier and never reach it. There was no warning because those requests never entered that code at all.
The fix moves the decision to the point both paths converge on. The more thorough approach is to make eligibility a flag on the request, but the blast radius is considerably larger and it was not attempted here.
As of July 27, 2026, this pull request remains open and unmerged; GitHub is authoritative for its live status.