Filter by type:
Filter by tags:
lfm2.5-1.2b-instruct
Try LFM • Docs • LEAP • Discord
# LFM2.5-1.2B-Instruct
LFM2.5 is a new family of hybrid models designed for **on-device deployment**. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **Best-in-class performance**: A 1.2B model rivaling much larger models, bringing high-quality AI to your pocket.
- **Fast edge inference**: 239 tok/s decode on AMD CPU, 82 tok/s on mobile NPU. Runs under 1GB of memory with day-one support for llama.cpp, MLX, and vLLM.
- **Scaled training**: Extended pre-training from 10T to 28T tokens and large-scale multi-stage reinforcement learning.
Find more information about LFM2.5 in our blog post.
## 🗒️ Model Details
LFM2.5-1.2B-Instruct is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
lfm2.5-8b-a1b
Try LFM •
Docs •
LEAP •
Discord
# LFM2.5-8B-A1B
LFM2.5 is a new family of hybrid models designed for on-device deployment. It builds on the LFM2 architecture with extended pre-training and reinforcement learning.
- **On-device personal assistant**: Designed to power real-life applications, chaining tool calls, and following complex instructions on all devices.
- **Compressed performance**: Competitive with much larger dense and MoE models on instruction following and agentic tasks.
- **Unmatched throughput**: Fastest in its size class on both CPU and GPU inference, with day-one support for llama.cpp, MLX, vLLM, and SGLang.
Find more information about LFM2.5-8B-A1B in our blog post.
**AA-Omniscience Index (higher is better) rewards correct answers and penalizes hallucinations. Scores range from -100 to 100. See more results on Artificial Analysis.*
## 🗒️ Model Details
LFM2.5-8B-A1B is a general-purpose text-only model with the following features:
...
Repository: localaiLicense: other
ced-base-f16
CED (Consistent Ensemble Distillation, Xiaomi) is a sound-event classifier that tags everyday sounds (baby cry, footsteps, glass breaking, alarms, dog bark, ...) into the 527-class AudioSet ontology. This is the f16 GGUF for the ced backend (a standalone C++/ggml port). Recommended default: fastest on CPU and near-lossless. Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-tiny-f16
CED-tiny (5.5M params, Pi-class / edge) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-mini-f16
CED-mini (9.6M params, low-power) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
ced-small-f16
CED-small (22M params, balanced size/accuracy) sound-event classifier over the 527-class AudioSet ontology (baby cry, footsteps, glass breaking, alarms, dog bark, ...). f16 GGUF for the ced backend (recommended (fastest on CPU)). Use POST /v1/audio/classification, or the realtime websocket API for live recognition.
Repository: localaiLicense: apache-2.0
longcat-video-avatar-1.5
LongCat-Video-Avatar-1.5 served by LocalAI's dedicated CUDA backend. Turns
speech plus a prompt into an avatar video, optionally conditioning on a
portrait, and continues across multiple segments for longer audio.
Avatar generation also loads tokenizer, text encoder, and VAE components
from LongCat-Video. Plan for very large downloads and substantial NVIDIA
GPU or unified memory; CPU and macOS execution are unsupported.
Repository: localaiLicense: mit
qwen3-coder-30b-a3b-vllm-cpp
Qwen3-Coder-30B-A3B on vllm.cpp: a coding and agentic-tool-use model, 30B
total parameters with about 3B active per token, gated token-exact against
vLLM on this engine.
The tool-call parser is named explicitly rather than auto-detected, and that
matters here. Qwen3-Coder's tool dialect is byte-identical on the wire to
another family's, so template sniffing cannot separate the two and would fall
back to the wrong parser. With qwen3_coder named, tool calls arrive as real
tool_calls on the OpenAI response.
This is the bf16 checkpoint, roughly 57 GB of weights, which is what the
engine was gated on. Being bf16 rather than NVFP4 it does not need Blackwell
on its own account, but LocalAI's CUDA images for this backend are currently
built for Blackwell-family GPUs only, so on an older card use the CPU build.
Repository: localaiLicense: apache-2.0
qwen3-4b-vllm-cpp
Qwen3-4B on vllm.cpp, in bf16. The small end of the engine's gated dense
family, which reaches parity with vLLM on every axis at concurrency 1.
bf16 rather than NVFP4 on purpose: this is the entry that runs where the
flagship NVFP4 checkpoints cannot, including Apple Silicon via Metal, Vulkan
and plain CPU. Roughly 8 GB of weights, plus about 4.5 GB of KV cache at the
context configured here.
Tool calling and the thinking split are parsed inside the engine.
Repository: localaiLicense: apache-2.0
qwen3-0.6b-vllm-cpp
Qwen3-0.6B on vllm.cpp, in bf16. Roughly 1.4 GB of weights, which makes it
the cheapest way to confirm a vllm-cpp install actually serves before
committing disk and memory to one of the large checkpoints.
It runs anywhere the backend does, CPU included, and it is a real chat model
rather than a stub, so tool calling and the thinking split can be exercised
on it too.
Repository: localaiLicense: apache-2.0
moss-tts-cpp-v1_5-q8_0
MOSS-TTS-Local v1.5 (C++/ggml, moss-tts.cpp), Q8_0 (~6 GB), near-lossless. Native
C++ text-to-speech for the OpenMOSS MOSS-TTS-Local Transformer v1.5 (a GPT-J local
transformer decoded through the MOSS-Audio-Tokenizer-v2 neural codec), 48kHz stereo
output with reference-audio voice cloning (set `voice` to a reference .wav). Verified
per component against the reference PyTorch and about 2x faster per frame on CPU.
Repository: localaiLicense: apache-2.0
allenai_olmo-3.1-32b-think
The **Olmo-3.1-32B-Think** model is a large language model (LLM) optimized for efficient inference using quantized versions. It is a quantized version of the original **allenai/Olmo-3.1-32B-Think** model, developed by **bartowski** using the **imatrix** quantization method.
### Key Features:
- **Base Model**: `allenai/Olmo-3.1-32B-Think` (unquantized version).
- **Quantized Versions**: Available in multiple formats (e.g., `Q6_K_L`, `Q4_1`, `bf16`) with varying precision (e.g., Q8_0, Q6_K_L, Q5_K_M). These are derived from the original model using the **imatrix calibration dataset**.
- **Performance**: Optimized for low-memory usage and efficient inference on GPUs/CPUs. Recommended quantization types include `Q6_K_L` (near-perfect quality) or `Q4_K_M` (default, balanced performance).
- **Downloads**: Available via Hugging Face CLI. Split into multiple files if needed for large models.
- **License**: Apache-2.0.
### Recommended Quantization:
- Use `Q6_K_L` for highest quality (near-perfect performance).
- Use `Q4_K_M` for balanced performance and size.
- Avoid lower-quality options (e.g., `Q3_K_S`) unless specific hardware constraints apply.
This model is ideal for deploying on GPUs/CPUs with limited memory, leveraging efficient quantization for practical use cases.
Repository: localaiLicense: apache-2.0
lfm2-1.2b
LFM2-1.2B is a hybrid liquid model designed for edge AI and on-device deployment, offering fast inference and multilingual support across 8 languages. It's optimized for agentic tasks, data extraction, and multi-turn conversations with efficient CPU/GPU/NPU compatibility.
Repository: localaiLicense: lfm1.0
insightface-buffalo-s
Small insightface pack (SCRFD-500MF detector + MBF 512-d embedder +
genderage, ~159MB). Good fit for mid-range CPU deployments.
NON-COMMERCIAL RESEARCH USE ONLY.
Repository: localaiLicense: insightface-non-commercial
insightface-opencv-int8
Int8-quantized OpenCV Zoo face pair (YuNet int8 + SFace int8, ~12MB).
Roughly 3x smaller and noticeably faster on CPU than the fp32 variant
at comparable accuracy for face tasks. APACHE 2.0 — commercial-safe.
Weights are downloaded on install via LocalAI's gallery mechanism.
Repository: localaiLicense: apache-2.0
face-detect-buffalo-s
Face recognition with insightface's `buffalo_s` pack (SCRFD-500MF
detector + MBF 512-d embedder), converted to a C++/ggml GGUF for the
`face-detect` backend. Small and CPU-friendly: a good fit for
mid-range and edge deployments.
The architecture (`facedetect.arch`) is read from the GGUF metadata,
so this entry alone selects the buffalo_s engine. This GGUF also
embeds the MiniFASNet anti-spoof ensemble, available via the
FaceVerify `anti_spoof` request flag. NON-COMMERCIAL RESEARCH USE
ONLY.
Repository: localaiLicense: insightface-non-commercial
wespeaker-resnet34
Speaker recognition with WeSpeaker's ResNet34 trained on VoxCeleb,
exported to ONNX. 256-d embeddings, CPU-friendly — avoids the
PyTorch runtime entirely (onnxruntime only). APACHE 2.0.
Pair with the `speaker-recognition` backend's OnnxDirectEngine.
Use when ECAPA-TDNN's torch dependency is undesirable (small
images, edge deployments).
Repository: localaiLicense: cc-by-4.0
voice-detect-wespeaker-resnet34
Speaker recognition with WeSpeaker's ResNet34 trained on VoxCeleb,
converted to a C++/ggml GGUF for the `voice-detect` backend. 256-d
embeddings, CPU-friendly and runtime-free (no onnxruntime or torch).
CC-BY-4.0.
Use when you want WeSpeaker's ResNet34 topology instead of
ECAPA-TDNN. The embedding architecture (`voicedetect.arch`) is read
from the GGUF metadata, so this entry alone selects the engine.
Repository: localaiLicense: cc-by-4.0
voice-detect-campplus
Speaker recognition with 3D-Speaker's CAM++ trained on VoxCeleb,
converted to a C++/ggml GGUF for the `voice-detect` backend. 192-d
embeddings, a fast context-aware masking topology well-suited to
CPU and edge deployments. APACHE 2.0.
The embedding architecture (`voicedetect.arch`) is read from the
GGUF metadata, so this entry alone selects the CAM++ engine.
Repository: localaiLicense: apache-2.0
rfdetr-cpp-nano
RF-DETR Nano object detection model, served via the native rfdetr.cpp backend (ggml + purego, no Python).
Q8_0 quantization is the recommended default for CPU: same accuracy as F16/F32, ~20MB on disk, fastest CPU latency.
Pure C++/ggml runtime; no Python dependencies. Drop-in for the /v1/detection endpoint.
Repository: localaiLicense: apache-2.0
locate-anything-3b
NVIDIA LocateAnything-3B open-vocabulary object detection (visual grounding), served via the native
locate-anything.cpp backend (C++/ggml + purego, no Python). Describe what to find in a text prompt and
get labeled boxes back; separate multiple categories with . Q8_0 is the recommended default:
box-identical to F16/F32, ~6.3GB, fastest CPU latency. Drop-in for the /v1/detection endpoint (pass the
prompt).
Repository: localaiLicense: other
Page 1