Filter by type:
Filter by tags:
ling-3.0-flash-iq1
Ling-3.0-flash is InclusionAI's MIT-licensed hybrid reasoning MoE model
with 124B total parameters and 5.5B active parameters per token. It targets
coding, deep research, instruction following, and agentic workflows with a
native 256K-token context window.
This default entry uses the 36.5 GB AD-IQ1_M GGUF. A higher-quality
44.7 GB AD-IQ2_XS model is available as a variant.
Repository: localaiLicense: mit
inkling
# Inkling
BF16 |
NVFP4 |
Playground |
Tinker Cookbook |
Acceptable Use
## 1. General Information
Inkling is a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. It is intended for use in English and other languages, and across multiple coding languages. The model is designed to be used by developers building AI-powered applications, including agentic and tool-use systems, coding assistants, chatbots, and retrieval-augmented generation systems, and is suitable for general-purpose conversational use, instruction-following, and other natural language and multimodal tasks. It is released with open weights to support research, fine-tuning and integration into third-party products by downstream developers.
**Languages:** English, with general multilingual capabilities across other languages.
## 2. Getting Started
Try Inkling on the Tinker Playground or access via API using the Tinker Cookbook.
Inkling supports local deployment using the following open-source libraries:
* SGLang (recipe, PR)
* vLLM (recipe, PR)
* TokenSpeed (recipe, PR)
* Unsloth (recipe, PR)
* Huggingface (recipe, PR)
...
Repository: localaiLicense: apache-2.0
qwythos-9b-v2
Empero AI
# Qwythos-9B-v2 — the new and improved Qwythos
The next iteration of Qwythos: **all the reasoning of Qwythos-9B, with the looping behavior fixed.** v2 keeps the deep chain-of-thought, the uncensored research posture, and the 1M-token context of its predecessor, and cleans up the rough edges that showed up in real use.
- 🔁 **Looping behavior eliminated** — repetition/degeneration under greedy or low-temperature decoding dropped from **6.7% → 0%**. You can serve it *without* leaning on `repetition_penalty` as a band-aid.
- 🧠 **Reasoning fully preserved** — MMLU, GSM8K, GPQA, ARC and HumanEval are all held at (or above) the v1 level. This is a *hygiene* upgrade, not a capability regression.
- 🧩 **MTP head restored** — the native multi-token-prediction module (dropped in the previous export) is back, so config and weights agree and speculative-decoding setups work.
- 🪪 **Cleaner identity** — the model no longer prefaces unrelated answers with its identity; it introduces itself only when you actually ask.
- 🔓 **Still intentionally uncensored** for research, cybersecurity, red-teaming, biology, chemistry, pharmacology and clinical work.
- 📜 **St
...
Repository: localaiLicense: apache-2.0
agents-a1-4b
Agents-A1-4B is InternScience's Apache-2.0 dense 4B agentic model, based on
Qwen3.5. It is trained for long-horizon search, engineering and scientific
research, instruction following, tool use, and multimodal tasks. This entry
uses the official Q4_K_M GGUF quantization and vision projector.
Repository: localaiLicense: apache-2.0
agents-a1-4b-q8
Agents-A1-4B is InternScience's Apache-2.0 dense 4B agentic model, based on
Qwen3.5. It is trained for long-horizon search, engineering and scientific
research, instruction following, tool use, and multimodal tasks. This entry
uses the official Q8_0 GGUF quantization and vision projector.
Repository: localaiLicense: apache-2.0
bigbang-v1-q4-k-m
BigBang-v1 is an Apache-2.0 multimodal reasoning model fine-tuned from
Qwen3.6-35B-A3B for scientific research, coding, long-horizon search, and
tool use. It supports image input, a 262K native context window, and
built-in multi-token prediction. This entry uses the recommended Q4_K_M
GGUF quantization and the F16 vision projector.
Repository: localaiLicense: apache-2.0
bigbang-v1-q6-k
BigBang-v1 is an Apache-2.0 multimodal reasoning model fine-tuned from
Qwen3.6-35B-A3B for scientific research, coding, long-horizon search, and
tool use. It supports image input, a 262K native context window, and
built-in multi-token prediction. This entry uses the higher-quality Q6_K
GGUF quantization and the F16 vision projector.
Repository: localaiLicense: apache-2.0
gemma-4-26b-a4b-it-qat
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
> [!Note]
> This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model.
> Four versions of the QAT checkpoints are available:
> * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models.
> * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B.
> * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B.
> * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B
...
Repository: localaiLicense: apache-2.0
gemma-4-12b-it-qat-q4_0
Hugging Face |
GitHub |
Launch Blog |
Documentation
License: Apache 2.0 | Authors: Google DeepMind
> [!Note]
> This model card is for the new versions of the Gemma 4 family optimized with Quantization-Aware Training (QAT), which allows preserving similar quality to bfloat16 while dramatically reducing the memory requirements to load the model.
> Four versions of the QAT checkpoints are available:
> * **Unquantized QAT checkpoints** (Q4_0): Half-precision weights extracted from the QAT pipeline, ideal for custom downstream compilation and research. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B, and their drafter models.
> * **GGUF** (Q4_0): Ready-to-deploy formats for broad ecosystem compatibility. Available for Gemma 4 E2B, E4B, 12B, 26B A4B, and 31B.
> * **Mobile-optimized** (wNa8o8): A custom schema engineered explicitly for mobile hardware efficiency. It features targeted 2-bit decoding layers, optimized KV caches, and static activations to maximize VRAM savings. Available for Gemma 4 E2B and E4B.
> * **Compressed Tensors** (w4a16): QAT checkpoints serialized in the compressed-tensors format for native, optimized inference with vLLM. Available for Gemma 4 E2B, E4B, 12B
...
Repository: localaiLicense: apache-2.0
zero-gemma4-e4b-openzero-q5-k-m
Zero Gemma4-E4B OpenZero is a Gemma 4 E4B instruction fine-tune for
local coding, research, chat, and agentic workflows. It was trained on
2,033 curated OpenZero examples and is distributed as one merged Q5_K_M
GGUF file.
License: OpenZero Community Source v1; see the model repository for terms.
Repository: localaiLicense: other
xyz-aquila-mini
XYZ-Aquila-mini is an Apache-2.0, open-weight thinking model based on
Qwen3.6-35B-A3B. It is tuned for agentic deep search, long-horizon planning,
bilingual web research, evidence aggregation, source verification, and
recovery from failed tool interactions. This entry uses the Q4_K_M GGUF;
the Q6_K variant offers higher fidelity on hosts with more memory.
The checkpoint supports Qwen-compatible reasoning and tool-call formats.
Search, scraping, and Python execution are supplied by the agent harness,
not by the model weights themselves.
Repository: localaiLicense: apache-2.0
chroma1-hd
Chroma1-HD is an 8.9B-parameter text-to-image foundation model derived from FLUX.1-schnell with reduced parameter count via architectural optimizations. Designed as a base for creators, researchers, and downstream fine-tuning. Recommended inference: 40 steps, CFG 3.0, bfloat16.
Repository: localaiLicense: apache-2.0
arex-turbo
AREX-Turbo is BAAI's compact 4B deep-research agent, fine-tuned from
Qwen3.5-4B for long-horizon search, evidence aggregation, constraint
verification, and tool-assisted reasoning. It supports text and image
input with a 262K-token context window. This entry uses the recommended
Q4_K_M GGUF quantization.
Repository: localaiLicense: apache-2.0
arex-turbo-q8
AREX-Turbo is BAAI's compact 4B deep-research agent, fine-tuned from
Qwen3.5-4B for long-horizon search, evidence aggregation, constraint
verification, and tool-assisted reasoning. It supports text and image
input with a 262K-token context window. This entry uses the higher-quality
Q8_0 GGUF quantization.
Repository: localaiLicense: apache-2.0
cajal-4b-p2pclaw
CAJAL-4B is a Qwen3.5-based 4B GGUF model specialized for scientific paper generation. It is tuned for academic structure, citation formats, LaTeX, and domain-specific technical writing within the P2PCLAW ecosystem, while remaining suitable for local llama-cpp inference.
Repository: localaiLicense: apache-2.0
nanbeige4.2-3b-q8
Nanbeige4.2-3B is a compact agentic and reasoning model with 3B non-embedding parameters. Its Looped Transformer architecture reuses transformer layers to increase model capacity without adding parameters. It targets tool use, coding, mathematics, scientific reasoning, office workflows, and deep research, with English and Chinese support and a context length of up to 262K tokens. This entry uses the Q8_0 GGUF quantization.
Repository: localaiLicense: apache-2.0
nanbeige4.2-3b
Nanbeige4.2-3B is a compact agentic and reasoning model with 3B non-embedding parameters. Its Looped Transformer architecture reuses transformer layers to increase model capacity without adding parameters. It targets tool use, coding, mathematics, scientific reasoning, office workflows, and deep research, with English and Chinese support and a context length of up to 262K tokens. This entry uses the Q4_K_M GGUF quantization.
Repository: localaiLicense: apache-2.0
z-image-diffusers
Z-Image is the foundation model of the ⚡️-Image family, engineered for good quality, robust generative diversity, broad stylistic coverage, and precise prompt adherence. While Z-Image-Turbo is built for speed, Z-Image is a full-capacity, undistilled transformer designed to be the backbone for creators, researchers, and developers who require the highest level of creative freedom.
Repository: localaiLicense: apache-2.0
mox-small-1-i1
The model, **vanta-research/mox-small-1**, is a small-scale text-generation model optimized for conversational AI tasks. It supports chat, persona research, and chatbot applications. The quantized versions (e.g., i1-Q4_K_M, i1-Q4_K_S) are available for efficient deployment, with the i1-Q4_K_S variant offering the best balance of size, speed, and quality. The model is designed for lightweight inference and is compatible with frameworks like HuggingFace Transformers.
Repository: localaiLicense: apache-2.0
liquidai_lfm2-1.2b-rag
Based on LFM2-1.2B, LFM2-1.2B-RAG is specialized in answering questions based on provided contextual documents, for use in RAG (Retrieval-Augmented Generation) systems.
Use cases:
Chatbot to ask questions about the documentation of a particular product.
Custom support with an internal knowledge base to provide grounded answers.
Academic research assistant with multi-turn conversations about research papers and course materials.
Repository: localaiLicense: lfm1.0
insightface-buffalo-l
Face recognition using insightface's `buffalo_l` pack
(SCRFD-10GF detector + ResNet50 ArcFace 512-d embedder + genderage head, ~326MB).
Default choice, highest accuracy.
Weights delivered via LocalAI's gallery mechanism (SHA-256 verified,
cached in the models directory like any other managed model).
NON-COMMERCIAL RESEARCH USE ONLY. For commercial use see `insightface-opencv`.
Repository: localaiLicense: insightface-non-commercial
Page 1