The model config architecture is listed in this runtime registry. This is family/architecture evidence only; exact checkpoint loading, model-specific configuration, hardware, and generation were not tested.
Checked the exact model config and the runtime architecture registry; no matching declared architecture was found. This is unknown, not evidence of incompatibility.
Checked the exact model config and the runtime architecture registry; no matching declared architecture was found. This is unknown, not evidence of incompatibility.
MLX community uploads may target mlx-vlm; an upload observation does not establish mlx-lm compatibility.
MEMORY ESTIMATES
Which file should I grab?
The summary estimate of about 3 GB is the recommended Q4_K_M quant (1.1 GB observed), from file size × 1.2 + 1.5 GB. Each card is the largest observed quant whose estimate fits that capacity.
About 4 GB using file size × 1.2 + 1.5 GB. Fit is an estimate; compatibility, speed, context length and hardware have not been tested.
Memory estimates are the observed file size plus fixed headroom (20% plus 1.5 GB runtime allowance); real needs also depend on context length, KV cache and runtime settings. Fit is an estimate; it does not establish runtime compatibility, speed, context length, or hardware performance. Text-weights-only estimates do not include a projector or total vision memory where unknown.
Terms: a quant is a lower-precision version of the weights; GGUF is a file format used by llama.cpp and compatible apps; MLX targets Apple silicon; vLLM is a model-serving engine for NVIDIA GPUs; the KV cache is stored attention state that grows with context length. Terms explained ↗
OBSERVED FILES
Quant uploads
Sizes and file groups come from the listed uploader repositories. A size is marked as an estimated fit only for a complete observed GGUF quant group.