Filter by type:
Filter by tags:
muse-glimmer-30b
Muse Glimmer is Meta Superintelligence Labs' Apache-2.0 dense 30B model for
autonomous agentic work, coding, tool use, long-horizon reasoning, and
multimodal understanding. It supports more than 100 languages, interleaved
text and image input through its 1.8B-parameter perception encoder, and a
131K-token context window.
This entry uses the publisher's higher-quality dynamic K-quant GGUF and
official quantized vision projector. Automatic variant selection can use
the smaller 17 GB quantization or a DFlash-accelerated build when it fits.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-dflash
Muse Glimmer's higher-quality dynamic K-quant GGUF with the official
quantized perception encoder and DFlash drafter. DFlash proposes blocks of
up to 16 tokens for the target to verify in parallel, accelerating output
without changing model quality. Flash attention is enabled for this path.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder. It preserves the model's agentic, coding, tool-use,
multilingual, and image-understanding capabilities for hosts with less
memory than the dynamic quantization requires.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb-dflash
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder and DFlash drafter. This is the lowest-memory published
build that retains image understanding and block-speculative decoding.
Flash attention is enabled for the DFlash path.
Repository: localaiLicense: apache-2.0
gemmable-4-12b-mtp
## Gemmable 4 12B
Gemmable 4 12B is a GGUF export of Gemma 4 12B fine-tuned on Fable-5 style
reasoning and assistant traces.
## Highlights
- Base model: `google/gemma-4-12B`
- Format: GGUF
- Training style: Fable-5 style reasoning and assistant traces
- Distribution: fp16 GGUF plus matching assistant GGUFs for each quant
- Intended use: local inference, coding, reasoning, and assistant workflows
## How to use
### llama.cpp
Standard load:
```bash
llama-server -m "gemmable-4-12b-fp16.gguf"
```
Speculative / draft-MTP load:
```bash
llama-server -m "gemmable-4-12b-Q4_K_M.gguf" \
--spec-draft-model "gemmable-4-12b-Q4_K_M-mtp.gguf" \
--spec-type draft-mtp \
--spec-draft-n-max 4
```
Use the matching fp16 or quantized main file with its `-mtp` companion.
### LM Studio
1. Search this repo, download target + mtp file.
2. Load target.
3. Load settings → Speculative Decoding → select mtp file file.
(Requires a llama.cpp runtime with Gemma 4 MTP support from ggml-org/llama.cpp#23398. LocalAI's pinned llama.cpp backend already carries it, so this entry runs draft-mtp out of the box.)
## GGUF / local inference notes
...
Repository: localai
laguna-s-2.1-q8
Laguna S 2.1 is Poolside's 118B-parameter, 8B-active Mixture-of-Experts model for agentic software engineering. It supports tool use and a native one-million-token context window; the official GGUF recommends 256K context for best output quality. This entry uses the 129 GB Q8_0 build, with routed experts quantized to Q8_0 and the signal path kept in BF16.
License: OpenMDW 1.1.
Repository: localaiLicense: other
laguna-s-2.1
Laguna S 2.1 is Poolside's 118B-parameter, 8B-active Mixture-of-Experts model for agentic software engineering. It supports tool use and a native one-million-token context window; the official GGUF recommends 256K context for best output quality. This default entry uses the current 96 GB Q4_K_M artifact, with imatrix-quantized routed experts and a Q8_0 signal path.
License: OpenMDW 1.1.
Repository: localaiLicense: other
gemma-4-26b-a4b-it-qat
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
> [!Note]
> This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model.
> Four versions of the QAT checkpoints are available:
> * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models.
> * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B.
> * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B.
> * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B
...
Repository: localaiLicense: apache-2.0
gemma-4-12b-it-qat-q4_0
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
> [!Note]
> This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model.
> Four versions of the QAT checkpoints are available:
> * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models.
> * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B.
> * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B.
> * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B
...
Repository: localaiLicense: apache-2.0
nemotron-3-nano-omni-30b-a3b-reasoning-apex
# Model Overview
### Description:
NVIDIA Nemotron 3 Nano Omni is a multimodal large language model that unifies video, audio, image, and text understanding to support enterprise-grade Q&A, summarization, transcription, and document intelligence workflows. It extends the Nemotron Nano family with integrated video+speech comprehension, Graphical User Interface (GUI), Optical Character Recognition (OCR), and speech transcription capabilities, enabling end-to-end processing of rich enterprise content such as meeting recordings, M&E assets, training videos, and complex business documents. NVIDIA Nemotron 3 Nano Omni was developed by NVIDIA as part of the Nemotron model family.
This model is available for commercial use.
This model was improved using Qwen3-VL-30B-A3B-Instruct, Qwen3.5-122B-A10B, Qwen3.5-397B-A17B, Qwen2.5-VL-72B-Instruct, and gpt-oss-120b. For more information, please see the Training Dataset section below.
### License/Terms of Use
Governing Terms: Use of this model is governed by the NVIDIA Open Model Agreement
### Deployment Geography:
Global
...
Repository: localaiLicense: other
arex-turbo
AREX-Turbo is BAAI's compact 4B deep-research agent, fine-tuned from
Qwen3.5-4B for long-horizon search, evidence aggregation, constraint
verification, and tool-assisted reasoning. It supports text and image
input with a 262K-token context window. This entry uses the recommended
Q4_K_M GGUF quantization.
Repository: localaiLicense: apache-2.0
arex-turbo-q8
AREX-Turbo is BAAI's compact 4B deep-research agent, fine-tuned from
Qwen3.5-4B for long-horizon search, evidence aggregation, constraint
verification, and tool-assisted reasoning. It supports text and image
input with a 262K-token context window. This entry uses the higher-quality
Q8_0 GGUF quantization.
Repository: localaiLicense: apache-2.0
ui-mate-9b
UI-Mate-9B is Tencent's 9B-parameter multimodal computer-use agent,
fine-tuned from Qwen3.5-9B. It accepts task instructions, screenshots,
and interaction history, then emits reasoning and structured mouse and
keyboard actions for long-running desktop tasks. The model requires an
external runtime to execute its actions and should run with human
confirmation for sensitive operations. This entry uses the recommended
Q4_K_M GGUF quantization.
Repository: localaiLicense: apache-2.0
ui-mate-9b-q8
UI-Mate-9B is Tencent's 9B-parameter multimodal computer-use agent,
fine-tuned from Qwen3.5-9B. It accepts task instructions, screenshots,
and interaction history, then emits reasoning and structured mouse and
keyboard actions for long-running desktop tasks. The model requires an
external runtime to execute its actions and should run with human
confirmation for sensitive operations. This entry uses the higher-quality
Q8_0 GGUF quantization.
Repository: localaiLicense: apache-2.0
fara1.5-4b
Fara1.5-4B is Microsoft's 4B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-4B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the recommended Q4_K_M GGUF quantization.
Repository: localaiLicense: mit
fara1.5-4b-q8
Fara1.5-4B is Microsoft's 4B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-4B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the higher-quality Q8_0 GGUF quantization.
Repository: localaiLicense: mit
fara1.5-9b
Fara1.5-9B is Microsoft's 9B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-9B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the recommended Q4_K_M GGUF quantization.
Repository: localaiLicense: mit
fara1.5-9b-q8
Fara1.5-9B is Microsoft's 9B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-9B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the higher-quality Q8_0 GGUF quantization.
Repository: localaiLicense: mit
fara1.5-27b
Fara1.5-27B is Microsoft's 27B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-27B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the Q4_K_M GGUF quantization.
Repository: localaiLicense: mit
fara1.5-27b-q8
Fara1.5-27B is Microsoft's 27B-parameter multimodal computer-use agent for web browsers, fine-tuned from Qwen3.5-27B. It accepts screenshots and text, emits structured browser actions, supports a 262K-token context, and should be deployed with appropriate sandboxing and user-confirmation controls. This entry uses the Q8_0 GGUF quantization.
Repository: localaiLicense: mit
qwen_qwen3.5-35b-a3b
Qwen3.5-35B-A3B is a quantized multimodal language model with 35B parameters using an A3B MoE architecture. It supports image-text understanding and chat interactions via llama-cpp backend.
Repository: localaiLicense: apache-2.0
Page 1