The model config architecture is listed in this runtime registry. This is family/architecture evidence only; exact checkpoint loading, model-specific configuration, hardware, and generation were not tested.
The model config architecture is listed in this runtime registry. This is family/architecture evidence only; exact checkpoint loading, model-specific configuration, hardware, and generation were not tested.
MLX community uploads may target mlx-vlm; an upload observation does not establish mlx-lm compatibility.
MEMORY ESTIMATES
Which file should I grab?
The summary estimate of about 28 GB is the recommended Q4_K_M quant (21.7 GB observed), from file size × 1.2 + 1.5 GB. Each card is the largest observed quant whose estimate fits that capacity.
16 GB capacity estimate
Which file should I grab?
doesn't fit
No observed complete quant fits this capacity using file size × 1.2 + 1.5 GB.
24 GB capacity estimate
Which file should I grab?
doesn't fit
No observed complete quant fits this capacity using file size × 1.2 + 1.5 GB.
About 28 GB using file size × 1.2 + 1.5 GB. Fit is an estimate; compatibility, speed, context length and hardware have not been tested.
Memory estimates are the observed file size plus fixed headroom (20% plus 1.5 GB runtime allowance); real needs also depend on context length, KV cache and runtime settings. Fit is an estimate; it does not establish runtime compatibility, speed, context length, or hardware performance. Text-weights-only estimates do not include a projector or total vision memory where unknown.
Terms: a quant is a lower-precision version of the weights; GGUF is a file format used by llama.cpp and compatible apps; MLX targets Apple silicon; vLLM is a model-serving engine for NVIDIA GPUs; the KV cache is stored attention state that grows with context length. Terms explained ↗
OBSERVED FILES
Quant uploads
Sizes and file groups come from the listed uploader repositories. A size is marked as an estimated fit only for a complete observed GGUF quant group.
Answer checks: 4 of 4 prompts passed (quick pass/fail sanity checks, not a quality benchmark).
Text only: the vision projector (mmproj) file was not loaded, so image input was not tested.
Thinking off (enable_thinking=false).
One machine, one run set: temperature 0, 8K context, 1 warm-up plus 3 runs per prompt. Short prompt is 23 tokens; long prompt is 1,431 tokens (each model counts with its own tokenizer). Weights were loaded from an external USB drive with mmap off; load time is not published for this run. Your speed will differ with hardware, context length and settings.