Filter by type:
Filter by tags:
longcat-video
LongCat-Video served by LocalAI's dedicated CUDA backend. Generates video
from a text prompt or a start image. The SDPA attention path works without
FlashAttention and is suitable for CUDA 13 ARM64 systems such as DGX Spark.
This is a very large checkpoint (roughly 83 GB in Hugging Face storage) and
requires Linux with an NVIDIA CUDA GPU plus substantial memory and disk.
Repository: localaiLicense: mit
longcat-video-avatar-1.5
LongCat-Video-Avatar-1.5 served by LocalAI's dedicated CUDA backend. Turns
speech plus a prompt into an avatar video, optionally conditioning on a
portrait, and continues across multiple segments for longer audio.
Avatar generation also loads tokenizer, text encoder, and VAE components
from LongCat-Video. Plan for very large downloads and substantial NVIDIA
GPU or unified memory; CPU and macOS execution are unsupported.
Repository: localaiLicense: mit
qwen3-coder-30b-a3b-vllm-cpp
Qwen3-Coder-30B-A3B on vllm.cpp: a coding and agentic-tool-use model, 30B
total parameters with about 3B active per token, gated token-exact against
vLLM on this engine.
The tool-call parser is named explicitly rather than auto-detected, and that
matters here. Qwen3-Coder's tool dialect is byte-identical on the wire to
another family's, so template sniffing cannot separate the two and would fall
back to the wrong parser. With qwen3_coder named, tool calls arrive as real
tool_calls on the OpenAI response.
This is the bf16 checkpoint, roughly 57 GB of weights, which is what the
engine was gated on. Being bf16 rather than NVFP4 it does not need Blackwell
on its own account, but LocalAI's CUDA images for this backend are currently
built for Blackwell-family GPUs only, so on an older card use the CPU build.
Repository: localaiLicense: apache-2.0
qwen3-tts-llamacpp
Qwen3-TTS 1.7B Base served by the llama.cpp backend, using upstream's own
GGUF conversion. Runs on the full llama-cpp accelerator matrix (CUDA, ROCm,
SYCL, Vulkan, Metal). Streaming output and zero-shot voice cloning: set
`voice` to a reference clip or a saved Voice Library profile, which is
required since the Base checkpoint has no built-in speaker. 24kHz mono,
10 languages. Q8_0 backbone (~1.8 GB) plus a Q8_0 projector.
Repository: localaiLicense: apache-2.0
llm-compiler-13b-imat
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
llm-compiler-13b-ftd
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
llm-compiler-7b-imat-GGUF
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
llm-compiler-7b-ftd-imat
LLM Compiler is a state-of-the-art LLM that builds upon Code Llama with improved performance for code optimization and compiler reasoning.
LLM Compiler is free for both research and commercial use.
LLM Compiler is available in two flavors:
LLM Compiler, the foundational models, pretrained on over 500B tokens of LLVM-IR, x86_84, ARM, and CUDA assembly codes and trained to predict the effect of LLVM optimizations;
and LLM Compiler FTD, which is further fine-tuned to predict the best optimizations for code in LLVM assembly to reduce code size, and to disassemble assembly code to LLVM-IR.
Repository: localaiLicense: other
deepseek-v4-flash-q2
DeepSeek V4 Flash (IQ2XXS GGUF, ~81 GB) - only loadable via the ds4 backend.
Requires >=128 GB RAM. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q2-q4
DeepSeek V4 Flash (mixed q2/q4 GGUF, ~91 GB) - only loadable via the ds4 backend.
The last 6 expert layers are kept at Q4_K (the rest IQ2XXS), trading a little
extra memory for higher quality than the pure-q2 build while still fitting in
RAM on a 128 GB machine. imatrix-tuned. Metal (Darwin) or CUDA (Linux).
See https://github.com/antirez/ds4 for details.
Repository: localai
deepseek-v4-flash-q2-mtp
DeepSeek V4 Flash (IQ2XXS GGUF, ~81 GB) paired with the optional MTP
speculative-decoding weights (~3.5 GB) for a slight speedup. Only loadable
via the ds4 backend; requires >=128 GB RAM. MTP helps only with greedy
decoding (temperature 0), so the override pins temperature to 0. Metal
(Darwin) or CUDA (Linux). See https://github.com/antirez/ds4 for details.
Repository: localai