Filter by type:
Filter by tags:
ling-3.0-flash-iq1
Ling-3.0-flash is InclusionAI's MIT-licensed hybrid reasoning MoE model
with 124B total parameters and 5.5B active parameters per token. It targets
coding, deep research, instruction following, and agentic workflows with a
native 256K-token context window.
This default entry uses the 36.5 GB AD-IQ1_M GGUF. A higher-quality
44.7 GB AD-IQ2_XS model is available as a variant.
Repository: localaiLicense: mit
carbon-3b-q4
Carbon-3B is Hugging Face's 3B-parameter genomic foundation model for DNA
and RNA sequence generation, recovery, variant-effect prediction, and
motif-perturbation analysis. It supports 32,768 tokens natively and uses a
hybrid tokenizer with 6-mer DNA tokens.
This default entry uses the Q4_K_M GGUF. A higher-quality Q8_0 build is
available as a variant. Prefix DNA sequences with `` and use uppercase
A, C, G, and T characters in groups of six.
Repository: localaiLicense: apache-2.0
carbon-8b-q4
Carbon-8B is the largest model in Hugging Face's Carbon family of genomic
foundation models. It targets DNA and RNA sequence generation, recovery,
variant-effect prediction, and motif-perturbation analysis with a native
context length of 32,768 hybrid 6-mer DNA tokens.
This default entry uses the Q4_K_M GGUF. A higher-quality Q8_0 build is
available as a variant. Prefix DNA sequences with `` and use uppercase
A, C, G, and T characters in groups of six.
Repository: localaiLicense: apache-2.0
nemotron-3.5-lightning-30b-a3b-q4
NVIDIA Nemotron 3.5 Lightning is a text-only hybrid Mamba-2, attention, and
mixture-of-experts model with 30B total parameters and 3B active parameters.
It targets reasoning, coding, tool use, multilingual chat, and long-context
agent workflows, with a context window of up to one million tokens.
This entry uses the official Q4_K_M GGUF. Automatic variant selection can
choose the smaller NVFP4 build or the higher-quality Q8_0 build when it fits.
Repository: localaiLicense: openmdw-1.1
bonsai-27b-1bit
Bonsai 27B (PrismML) is a full 27B-class reasoning model in end-to-end 1-bit weights,
derived from the Qwen3.6-27B hybrid-attention backbone (~75% linear attention, 262K
context). At a true 1.125 bits/weight it deploys in ~3.9 GB (~14.2x smaller than FP16)
while retaining 89.5% of FP16 intelligence across 15 thinking-mode benchmarks (math
91.66, coding 81.88). Ships an optional 4-bit vision tower (mmproj) for image input,
included here.
The Q1_0_g128 weights and hybrid-attention kernels are only in the PrismML llama.cpp
fork, so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
ternary-bonsai-27b
Ternary Bonsai 27B (PrismML) is the quality-oriented operating point of the Bonsai 27B
family: full 27B-class reasoning in ternary {-1, 0, +1} weights on the Qwen3.6-27B
hybrid-attention backbone (262K context). At a true 1.71 bits/weight it deploys in
~7.2 GB (GGUF Q2_0_g128) and retains 95% of FP16 intelligence (80.49 average across 15
thinking-mode benchmarks) - a higher score than a conventional IQ2_XXS build at less
than two-thirds its footprint. Ships an optional 4-bit vision tower (mmproj), included.
The Q2_0 weights and hybrid-attention kernels are only in the PrismML llama.cpp fork,
so this runs on LocalAI's `bonsai` backend. A GPU is recommended. License: Apache 2.0.
Repository: localaiLicense: apache-2.0
deepseek-v4-flash
# DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
Technical Report👁️
## Introduction
We present a preview version of **DeepSeek-V4** series, including two strong Mixture-of-Experts (MoE) language models — **DeepSeek-V4-Pro** with 1.6T parameters (49B activated) and **DeepSeek-V4-Flash** with 284B parameters (13B activated) — both supporting a context length of **one million tokens**.
DeepSeek-V4 series incorporate several key upgrades in architecture and optimization:
1. **Hybrid Attention Architecture:** We design a hybrid attention mechanism combining Compressed Sparse Attention (CSA) and Heavily Compressed Attention (HCA) to dramatically improve long-context efficiency. In the 1M-token context setting, DeepSeek-V4-Pro requires only **27% of single-token inference FLOPs** and **10% of KV cache** compared with DeepSeek-V3.2.
2. **Manifold-Constrained Hyper-Connections (mHC):** We incorporate mHC to strengthen conventional residual connections, enhancing stability of signal propagation across layers while preserving model expressivity.
3. **Muon Optimizer:** We employ the Muon optimizer for faster convergence and greater training stability.
...
Repository: localaiLicense: mit
qwopus3.6-35b-a3b-coder-mtp
# 🌟 Qwopus3.6-35B-A3B-v1
## 💡 Base Model Overview
**Qwen3.6-35B-A3B** is an advanced hybrid sparse MoE (Mixture-of-Experts) model developed by Alibaba Cloud. It features 35B total parameters with only 3B active parameters per token, ensuring high inference efficiency. Architecturally, it combines Gated DeltaNet linear attention with standard gated attention layers, routing tokens across **256 experts**. It natively supports a massive **262k context window** and is specifically designed for high-performance agentic coding, deep reasoning, and multimodal tasks.
## 🚀 Model Refinement & Logic Tuning （Qwopus3.6-35B-A3B-v1）
🪐**Qwopus3.6-35B-A3B-v1** is a reasoning-enhanced MoE (Mixture of Experts) model fine-tuned on top of **Qwen3.6-35B-A3B**.
### 🛠 Training Strategy
The fine-tuning process for this model is structured into **three distinct stages of distributed SFT (Supervised Fine-Tuning)**, progressively scaling reasoning complexity and data diversity. This systematic approach ensures the model inherits the base MoE capabilities while sharpening its logic-handling depth.
...
Repository: localaiLicense: apache-2.0
lfm2.5-1.2b-instruct
Try LFM • Docs • LEAP • Discord
# LFM2.5-1.2B-Instruct
LFM2.5 is a new family of hybrid models designed for **on-device deployment**. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **Best-in-class performance**: A 1.2B model rivaling much larger models, bringing high-quality AI to your pocket.
- **Fast edge inference**: 239 tok/s decode on AMD CPU, 82 tok/s on mobile NPU. Runs under 1GB of memory with day-one support for llama.cpp, MLX, and vLLM.
- **Scaled training**: Extended pre-training from 10T to 28T tokens and large-scale multi-stage reinforcement learning.
Find more information about LFM2.5 in our blog post.
## 🗒️ Model Details
LFM2.5-1.2B-Instruct is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
lfm2.5-8b-a1b
Try LFM •
Docs •
LEAP •
Discord
# LFM2.5-8B-A1B
LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **On-device personal assistant**: Designed to power real-life applications, chaining tool calls, and following complex instructions on all devices.
- **Compressed performance**: Competitive with much larger dense and MoE models on instruction following and agentic tasks.
- **Unmatched throughput**: Fastest in its size class on both CPU and GPU inference, with day-one support for llama.cpp, MLX, vLLM, and SGLang.
Find more information about LFM2.5-8B-A1B in our blog post.
**AA-Omniscience Index (higher is better) rewards correct answers and penalizes hallucinations. Scores range from -100 to 100. See more results on Artificial Analysis.*
## 🗒️ Model Details
LFM2.5-8B-A1B is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
qwen3.6-35b-a3b-nvfp4-vllm-cpp
Qwen3.6-35B-A3B in NVFP4, served by vllm.cpp. A 35B mixture-of-experts model
with roughly 3B parameters active per token, so it reads like a much larger
model while costing about as much per token as a small one.
This is the engine's gated MoE checkpoint: token-for-token identical to vLLM
over the 315-prompt battery on both the synchronous and asynchronous paths,
at 0.92x to 0.97x vLLM's throughput from concurrency 1 to 32.
The architecture is a gated-delta-net hybrid, so automatic prefix caching is
off by default here where it would be on for a dense model. That is the
engine's own default and this entry does not override it.
Needs a Blackwell-class NVIDIA GPU and roughly 23 GB of weights plus KV cache.
Repository: localaiLicense: apache-2.0
ai21labs_ai21-jamba-reasoning-3b
AI21’s Jamba Reasoning 3B is a top-performing reasoning model that packs leading scores on intelligence benchmarks and highly-efficient processing into a compact 3B build.
The hybrid design combines Transformer attention with Mamba (a state-space model). Mamba layers are more efficient for sequence processing, while attention layers capture complex dependencies. This mix reduces memory overhead, improves throughput, and makes the model run smoothly on laptops, GPUs, and even mobile devices, while maintainig impressive quality.
Repository: localaiLicense: apache-2.0
lfm2-1.2b
LFM2-1.2B is a hybrid liquid model designed for edge AI and on-device deployment, offering fast inference and multilingual support across 8 languages. It's optimized for agentic tasks, data extraction, and multi-turn conversations with efficient CPU/GPU/NPU compatibility.
Repository: localaiLicense: lfm1.0
liquidai_lfm2-8b-a1b
LFM2 is a new generation of hybrid models developed by Liquid AI, specifically designed for edge AI and on-device deployment. It sets a new standard in terms of quality, speed, and memory efficiency.
We're releasing the weights of our first MoE based on LFM2, with 8.3B total parameters and 1.5B active parameters.
LFM2-8B-A1B is the best on-device MoE in terms of both quality (comparable to 3-4B dense models) and speed (faster than Qwen3-1.7B).
Code and knowledge capabilities are significantly improved compared to LFM2-2.6B.
Quantized variants fit comfortably on high-end phones, tablets, and laptops.
Repository: localaiLicense: lfm1.0
symiotic-14b-i1
SymbioticLM-14B is a state-of-the-art 17.8 billion parameter symbolic–transformer hybrid model that tightly couples high-capacity neural representation with structured symbolic cognition. Designed to match or exceed performance of top-tier LLMs in symbolic domains, it supports persistent memory, entropic recall, multi-stage symbolic routing, and self-organizing knowledge structures.
This model is ideal for advanced reasoning agents, research assistants, and symbolic math/code generation systems.
Repository: localaiLicense: afl-3.0
nousresearch_hermes-4-14b
Hermes 4 14B is a frontier, hybrid-mode reasoning model based on Qwen 3 14B by Nous Research that is aligned to you.
Read the Hermes 4 technical report here: Hermes 4 Technical Report
Chat with Hermes in Nous Chat: https://chat.nousresearch.com
Training highlights include a newly synthesized post-training corpus emphasizing verified reasoning traces, massive improvements in math, code, STEM, logic, creativity, and format-faithful outputs, while preserving general assistant quality and broadly neutral alignment.
What’s new vs Hermes 3
Post-training corpus: Massively increased dataset size from 1M samples and 1.2B tokens to ~5M samples / ~60B tokens blended across reasoning and non-reasoning data.
Hybrid reasoning mode with explicit … segments when the model decides to deliberate, and options to make your responses faster when you want.
Reasoning that is top quality, expressive, improves math, code, STEM, logic, and even creative writing and subjective responses.
Schema adherence & structured outputs: trained to produce valid JSON for given schemas and to repair malformed objects.
Much easier to steer and align: extreme improvements on steerability, especially on reduced refusal rates.
Repository: localaiLicense: apache-2.0
deepcogito_cogito-v1-preview-llama-70b
The Cogito LLMs are instruction tuned generative models (text in/text out). All models are released under an open license for commercial use.
Cogito models are hybrid reasoning models. Each model can answer directly (standard LLM), or self-reflect before answering (like reasoning models).
The LLMs are trained using Iterated Distillation and Amplification (IDA) - an scalable and efficient alignment strategy for superintelligence using iterative self-improvement.
The models have been optimized for coding, STEM, instruction following and general helpfulness, and have significantly higher multilingual, coding and tool calling capabilities than size equivalent counterparts.
In both standard and reasoning modes, Cogito v1-preview models outperform their size equivalent counterparts on common industry benchmarks.
Each model is trained in over 30 languages and supports a context length of 128k.
Repository: localaiLicense: llama3.1
nousresearch_deephermes-3-llama-3-3b-preview
DeepHermes 3 Preview is the latest version of our flagship Hermes series of LLMs by Nous Research, and one of the first models in the world to unify Reasoning (long chains of thought that improve answer accuracy) and normal LLM response modes into one model. We have also improved LLM annotation, judgement, and function calling.
DeepHermes 3 Preview is a hybrid reasoning model, and one of the first LLM models to unify both "intuitive", traditional mode responses and long chain of thought reasoning responses into a single model, toggled by a system prompt.
Hermes 3, the predecessor of DeepHermes 3, is a generalist language model with many improvements over Hermes 2, including advanced agentic capabilities, much better roleplaying, reasoning, multi-turn conversation, long context coherence, and improvements across the board.
The ethos of the Hermes series of models is focused on aligning LLMs to the user, with powerful steering capabilities and control given to the end user.
This is a preview Hermes with early reasoning capabilities, distilled from R1 across a variety of tasks that benefit from reasoning and objectivity. Some quirks may be discovered! Please let us know any interesting findings or issues you discover!
Repository: localaiLicense: llama3
deepcogito_cogito-v1-preview-llama-3b
The Cogito LLMs are instruction tuned generative models (text in/text out). All models are released under an open license for commercial use.
Cogito models are hybrid reasoning models. Each model can answer directly (standard LLM), or self-reflect before answering (like reasoning models).
The LLMs are trained using Iterated Distillation and Amplification (IDA) - an scalable and efficient alignment strategy for superintelligence using iterative self-improvement.
The models have been optimized for coding, STEM, instruction following and general helpfulness, and have significantly higher multilingual, coding and tool calling capabilities than size equivalent counterparts.
In both standard and reasoning modes, Cogito v1-preview models outperform their size equivalent counterparts on common industry benchmarks.
Each model is trained in over 30 languages and supports a context length of 128k.
Repository: localaiLicense: llama3.2
deepcogito_cogito-v1-preview-llama-8b
The Cogito LLMs are instruction tuned generative models (text in/text out). All models are released under an open license for commercial use.
Cogito models are hybrid reasoning models. Each model can answer directly (standard LLM), or self-reflect before answering (like reasoning models).
The LLMs are trained using Iterated Distillation and Amplification (IDA) - an scalable and efficient alignment strategy for superintelligence using iterative self-improvement.
The models have been optimized for coding, STEM, instruction following and general helpfulness, and have significantly higher multilingual, coding and tool calling capabilities than size equivalent counterparts.
In both standard and reasoning modes, Cogito v1-preview models outperform their size equivalent counterparts on common industry benchmarks.
Each model is trained in over 30 languages and supports a context length of 128k.
Repository: localaiLicense: llama3.1
nousresearch_hermes-4-70b
Hermes 4 70B is a frontier, hybrid-mode reasoning model based on Llama-3.1-70B by Nous Research that is aligned to you.
Read the Hermes 4 technical report here: Hermes 4 Technical Report
Chat with Hermes in Nous Chat: https://chat.nousresearch.com
Training highlights include a newly synthesized post-training corpus emphasizing verified reasoning traces, massive improvements in math, code, STEM, logic, creativity, and format-faithful outputs, while preserving general assistant quality and broadly neutral alignment.
What’s new vs Hermes 3
Post-training corpus: Massively increased dataset size from 1M samples and 1.2B tokens to ~5M samples / ~60B tokens blended across reasoning and non-reasoning data.
Hybrid reasoning mode with explicit … segments when the model decides to deliberate, and options to make your responses faster when you want.
Reasoning that is top quality, expressive, improves math, code, STEM, logic, and even creative writing and subjective responses.
Schema adherence & structured outputs: trained to produce valid JSON for given schemas and to repair malformed objects.
Much easier to steer and align: extreme improvements on steerability, especially on reduced refusal rates.
Repository: localaiLicense: llama3
Page 1