Filter by type:
Filter by tags:
ling-3.0-flash-iq1
Ling-3.0-flash is InclusionAI's MIT-licensed hybrid reasoning MoE model
with 124B total parameters and 5.5B active parameters per token. It targets
coding, deep research, instruction following, and agentic workflows with a
native 256K-token context window.
This default entry uses the 36.5 GB AD-IQ1_M GGUF. A higher-quality
44.7 GB AD-IQ2_XS model is available as a variant.
Repository: localaiLicense: mit
ling-3.0-flash-iq2
Ling-3.0-flash in the higher-quality 44.7 GB AD-IQ2_XS GGUF format. This
variant preserves more model fidelity for hosts with enough memory.
Repository: localaiLicense: mit
muse-glimmer-30b-dflash
Muse Glimmer's higher-quality dynamic K-quant GGUF with the official
quantized perception encoder and DFlash drafter. DFlash proposes blocks of
up to 16 tokens for the target to verify in parallel, accelerating output
without changing model quality. Flash attention is enabled for this path.
Repository: localaiLicense: apache-2.0
muse-glimmer-30b-17gb-dflash
Muse Glimmer's smaller 17 GB K-quant GGUF with the official quantized
perception encoder and DFlash drafter. This is the lowest-memory published
build that retains image understanding and block-speculative decoding.
Flash attention is enabled for the DFlash path.
Repository: localaiLicense: apache-2.0
deepseek-v4-flash-0731
# DeepSeek-V4-Flash-0731
Technical Report👁️
## Introduction
**DeepSeek-V4-Flash-0731** is the official release of **DeepSeek-V4-Flash**, superseding the preview version, with substantially enhanced agentic capabilities. It has the same model structure as DeepSeek-V4-Flash-DSpark, i.e. it comes with a speculative decoding module attached.
DeepSeek-V4-Flash-0731 outperforms DeepSeek-V4-Pro (Preview) on benchmarks listed below despite its far smaller activated parameter count, and is broadly competitive with the strongest proprietary models available.
Notes:
1. For the Code Agent tasks among the public benchmarks above, DeepSeek-V4-Flash-0731 is evaluated with the minimal mode of DeepSeek Harness (to be released) as the agent framework, using the `max` reasoning effort level with `temperature = 1.0, top_p = 0.95`.
2. † DSBench-FullStack is an internal full-stack development test set; DSBench-Hard is an internal test set of difficult coding-agent problems.
## Chat Template
...
Repository: localaiLicense: mit
deepseek-v4-flash
# DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Technical Report👁️
## Introduction
We present a preview version of **DeepSeek-V4** series, including two strong Mixture-of-Experts (MoE) language models — **DeepSeek-V4-Pro** with 1.6T parameters (49B activated) and **DeepSeek-V4-Flash** with 284B parameters (13B activated) — both supporting a context length of **one million tokens**.
DeepSeek-V4 series incorporate several key upgrades in architecture and optimization:
1. **Hybrid Attention Architecture:** We design a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to dramatically improve long-context efficiency. In the 1M-token context setting, DeepSeek-V4-Pro requires only **27% of single-token inference FLOPs** and **10% of KV cache** compared with DeepSeek-V3.2.
2. **Manifold-Constrained Hyper-Connections (mHC):** We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity.
3. **Muon Optimizer:** We employ the Muon optimizer for faster convergence and greater training stability.
...
Repository: localaiLicense: mit
laguna-s-2.1-dflash
Laguna S 2.1's 96 GB Q4_K_M target paired with Poolside's 2.2 GB BF16 DFlash drafter for speculative decoding. DFlash drafts a block of tokens per forward pass and accelerates generation without changing the target model's outputs. Flash attention is enabled because the DFlash path requires it.
License: OpenMDW 1.1.
Repository: localaiLicense: other
qwen3-4b-dflash
Qwen3-4B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend. This is the canonical DFlash pairing documented upstream (`z-lab/Qwen3-4B-DFlash` + `Qwen/Qwen3-4B`).
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3-4B target; the ~0.5 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3-4B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
qwen3.5-4b-dflash
Qwen3.5-4B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3.5-4B target; the ~0.6 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3.5-4B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
qwen3.5-9b-dflash
Qwen3.5-9B paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3.5-9B target; the ~1 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0 (Qwen3.5-9B target) / MIT (z-lab DFlash drafter).
Repository: localaiLicense: apache-2.0
qwen3.6-27b-dflash
Qwen3.6-27B (dense) paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend. DFlash gives its largest speedups on dense targets like this one.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The Q4_K_M file carries the full Qwen3.6-27B target; the ~1.8 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-dflash
Qwen3.6-35B-A3B (Mixture-of-Experts, ~3B active per token) paired with its DFlash block-diffusion drafter for speculative decoding on the llama.cpp backend. DFlash speedups on MoE targets are smaller than on dense models, but still useful.
DFlash produces a whole block of draft tokens in a single forward pass and injects the target model's hidden states into the drafter's attention, which keeps the drafter tiny while making drafting GPU-friendly. The UD-Q4_K_M file carries the full Qwen3.6-35B-A3B target; the ~0.4 GB Q8_0 drafter (`draft-dflash`) accelerates generation without changing the target's outputs. The drafter is not a standalone chat model: it only runs paired with the target, which is why both are bundled here.
Flash attention is required for DFlash and is enabled in this config. A GPU is recommended.
License: Apache 2.0.
Repository: localaiLicense: apache-2.0
step-3.7-flash
**[ModelPage]**: https://static.stepfun.com/blog/step-3.7-flash/
## 1. Introduction
Step 3.7 Flash is a 198B-parameter sparse Mixture-of-Experts (MoE) vision-language model that combines a 196B-parameter language backbone with a 1.8B-parameter vision encoder for native image understanding. Engineered for high-frequency production workloads, it activates approximately 11B parameters per token and delivers a throughput of up to 400 tokens per second. Step 3.7 Flash supports a 256k context window and offers three selectable reasoning levels (low, medium, and high) so developers can easily balance speed, cost, and cognitive depth.
We built Step 3.7 Flash for developers who need to scale agentic workflows that combine perception, search, and reasoning. It is designed to handle intensive tasks such as parsing massive financial reports in one pass, running multi-step search loops with cross-source verification, or operating concurrent coding agents in high-throughput pipelines.
## 2. Capabilities & Performance
### Multimodal Perception and Verification
...
Repository: localaiLicense: apache-2.0
qwen3.5-9b-deepseek-v4-flash
# Qwen3.5-9B
[](https://chat.qwen.ai)
> [!Note]
> This repository contains model weights and configuration files for the post-trained model in the Hugging Face Transformers format.
>
> These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers, etc.
Over recent months, we have intensified our focus on developing foundation models that deliver exceptional utility and performance. Qwen3.5 represents a significant leap forward, integrating breakthroughs in multimodal learning, architectural efficiency, reinforcement learning scale, and global accessibility to empower developers and enterprises with unprecedented capability and efficiency.
## Qwen3.5 Highlights
Qwen3.5 features the following enhancement:
- **Unified Vision-Language Foundation**: Early fusion training on multimodal tokens achieves cross-generational parity with Qwen3 and outperforms Qwen3-VL models across reasoning, coding, agents, and visual understanding benchmarks.
- **Efficient Hybrid Architecture**: Gated Delta Networks combined with sparse Mixture-of-Experts deliver high-throughput inference with minimal latency and cost overhead.
...
Repository: localaiLicense: apache-2.0
qwen3.6-27b-nvfp4-dflash-vllm-cpp
Qwen3.6-27B NVFP4 on vllm.cpp with DFlash block-diffusion speculative
decoding: the fastest configuration of this model the engine ships.
Where MTP drafts one token at a time, DFlash drafts a whole 16-token block in
a single non-autoregressive pass from a separate 3.5 GB drafter, then the
target verifies the block in one step. At concurrency 1 that measures 2.9x
the throughput of the same weights with speculation off, and at or above
vLLM's own DFlash-on decode.
Both checkpoints are installed for you: the target as a revision-pinned
snapshot, the drafter into models/Qwen3.6-27B-DFlash, which is where the
backend looks when speculative_config.model names it. The drafter shares the
target's embed_tokens and lm_head, so the two are not independently
swappable.
Needs a Blackwell-class NVIDIA GPU and roughly 28 GB of weights in total.
Repository: localaiLicense: apache-2.0
glm-4.7-flash-derestricted
This model is a quantized version of the original GLM-4.7-Flash-Derestricted model, derived from the base model `koute/GLM-4.7-Flash-Derestricted`. It is designed for restricted use, featuring tags like "derestricted," "uncensored," and "unlimited." The quantized versions (e.g., Q2_K, Q4_K_S, Q6_K) offer varying trade-offs between accuracy and efficiency, with the Q4_K_S and Q6_K variants being recommended for balanced performance. The model is optimized for fast inference and supports multiple quantization schemes, though some advanced quantization options (like IQ4_XS) are not available. It is intended for use in environments with specific constraints or restrictions.
Repository: localaiLicense: mit
huihui-glm-4.7-flash-abliterated-i1
The model is a quantized version of **huihui-ai/Huihui-GLM-4.7-Flash-abliterated**, optimized for efficiency and deployment. It uses GGUF files with various quantization levels (e.g., IQ1_M, IQ2_XXS, Q4_K_M) and is designed for tasks requiring low-resource deployment. Key features include:
- **Base Model**: Huihui-GLM-4.7-Flash-abliterated (unmodified, original model).
- **Quantization**: Supports IQ1_M to Q4_K_M, balancing accuracy and efficiency.
- **Use Cases**: Suitable for applications needing lightweight inference, such as edge devices or resource-constrained environments.
- **Downloads**: Available in GGUF format with varying quality and size (e.g., 0.2GB to 18.2GB).
- **Tags**: Abliterated, uncensored, and optimized for specific tasks.
This model is a modified version of the original GLM-4.7, tailored for deployment with quantized weights.
Repository: localaiLicense: mit
glm-4.7-flash
**GLM-4.7-Flash** is a 30B-A3B MoE (Model Organism Ensemble) model designed for efficient deployment. It outperforms competitors in benchmarks like AIME 25, GPQA, and τ²-Bench, offering strong accuracy while balancing performance and efficiency. Optimized for lightweight use cases, it supports inference via frameworks like vLLM and SGLang, with detailed deployment instructions in the official repository. Ideal for applications requiring high-quality text generation with minimal resource consumption.
Repository: localaiLicense: mit
deepseek-v4-flash-q2
DeepSeek V4 Flash (IQ2XXS GGUF, ~81 GB) - only loadable via the ds4 backend.
Requires >=128 GB RAM. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q2-q4
DeepSeek V4 Flash (mixed q2/q4 GGUF, ~91 GB) - only loadable via the ds4 backend.
The last 6 expert layers are kept at Q4_K (the rest IQ2XXS), trading a little
extra memory for higher quality than the pure-q2 build while still fitting in
RAM on a 128 GB machine. imatrix-tuned. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q4-ssd
DeepSeek V4 Flash (full 4-bit experts GGUF, ~153 GB) - only loadable via the
ds4 backend, with SSD streaming enabled so it runs on a 128 GB machine even
though the weights do not fit in RAM: routed MoE experts stream from the GGUF
on SSD while the non-routed weights stay resident. SSD streaming is Metal
(Darwin) only; generation speed depends on SSD speed and the expert cache.
Tune the routed-expert cache with the 'ssd_streaming_cache_experts:NGB'
option (default: automatic budget). See https://github.com/antirez/ds4.
Repository: localai
Page 1