MLX Block Streaming

Stream LARGE MODEL WEIGHTS through a CALLER-DECLARED MEMORY BUDGET without ever silently serving stale, placeholder, or leaked-bookkeeping weights in their place. Zero dependencies, pure Python over the standard library.

THE SILENT FAILURE. Streaming weights block by block means at any moment most of the model is not resident. The bookkeeping that decides which block is where is ordinary code -- a dict, an eviction policy, a refcount -- and when it is wrong it does not crash. It hands back A TENSOR OF THE RIGHT SHAPE containing the wrong numbers: a block that was evicted and not reloaded, a placeholder never overwritten, or one whose refcount leaked so it was reused while still in use.

The model then produces output. Slightly worse output. There is no exception, no NaN, and no obvious degradation -- inference on subtly wrong weights looks exactly like inference on a slightly worse model, which is indistinguishable from every other reason a model might underperform.

THE MEMORY BUDGET IS DECLARED BY YOU, not probed from the machine, because a budget inferred from available RAM changes between runs and makes the failure non-reproducible.

Every handle either resolves to the block it names or raises. There is no fallback path that returns something plausible.

VERIFIED: 61 tests, measured by running the suite.

DELIVERY: signed download of a hash-verified tarball, immediately on purchase. Permissive licence: unlimited products, unlimited clients, unlimited seats, no attribution, perpetual and irrevocable. One restriction, do not republish the source as source.