Filter by type:
Filter by tags:
longcat-video
LongCat-Video served by LocalAI's dedicated CUDA backend. Generates video
from a text prompt or a start image. The SDPA attention path works without
FlashAttention and is suitable for CUDA 13 ARM64 systems such as DGX Spark.
This is a very large checkpoint (roughly 83 GB in Hugging Face storage) and
requires Linux with an NVIDIA CUDA GPU plus substantial memory and disk.
Repository: localaiLicense: mit
longcat-video-avatar-1.5
LongCat-Video-Avatar-1.5 served by LocalAI's dedicated CUDA backend. Turns
speech plus a prompt into an avatar video, optionally conditioning on a
portrait, and continues across multiple segments for longer audio.
Avatar generation also loads tokenizer, text encoder, and VAE components
from LongCat-Video. Plan for very large downloads and substantial NVIDIA
GPU or unified memory; CPU and macOS execution are unsupported.
Repository: localaiLicense: mit
qwen3.6-27b-nvfp4-vllm-cpp
Qwen3.6-27B in NVFP4, served by vllm.cpp: LocalAI's own C++ port of vLLM, with
no Python at inference time. This is the reference text-generation checkpoint
the engine is gated on, token-for-token identical to vLLM's own greedy output
over the 235-prompt correctness battery, and measured at or above vLLM's
throughput at every concurrency from 1 to 32.
The weights are PINNED to revision 890bdef7. That pin is load-bearing, not
housekeeping: the same repository name was later re-quantized to FP8 W8A8
throughout, so an unpinned copy of this entry serves entirely different
weights with no error and none of the measured behaviour above.
Needs a Blackwell-class NVIDIA GPU (NVFP4 has no kernel on older
architectures) and roughly 25 GB of weights plus KV cache. Tool calling and
the thinking split are parsed inside the engine.
Repository: localaiLicense: apache-2.0
qwen3.6-27b-nvfp4-mtp-vllm-cpp
Qwen3.6-27B NVFP4 on vllm.cpp with MTP speculative decoding enabled.
MTP (Multi-Token Prediction) drafts from a head that ships inside the target
checkpoint's own mtp.* tensors, so there is no second model to download and
no extra weights to manage. The verifier accepts roughly 85% of drafted
tokens on prose and 92% on code, worth about 1.5x to 1.6x the decode
throughput of the same weights with speculation off, and it holds that lead
at concurrency 2, 4 and 8.
Same weights and same revision pin as qwen3.6-27b-nvfp4-vllm-cpp; install
that entry instead if you would rather not spend the extra memory. The
speculative state (a doubled recurrent-state slot plus the draft cache and
head) costs roughly 3.6 GB on top of the base footprint.
MTP here is depth 1 by construction: the engine refuses
num_speculative_tokens above 1 for this method.
Repository: localaiLicense: apache-2.0
qwen3.6-27b-nvfp4-dflash-vllm-cpp
Qwen3.6-27B NVFP4 on vllm.cpp with DFlash block-diffusion speculative
decoding: the fastest configuration of this model the engine ships.
Where MTP drafts one token at a time, DFlash drafts a whole 16-token block in
a single non-autoregressive pass from a separate 3.5 GB drafter, then the
target verifies the block in one step. At concurrency 1 that measures 2.9x
the throughput of the same weights with speculation off, and at or above
vLLM's own DFlash-on decode.
Both checkpoints are installed for you: the target as a revision-pinned
snapshot, the drafter into models/Qwen3.6-27B-DFlash, which is where the
backend looks when speculative_config.model names it. The drafter shares the
target's embed_tokens and lm_head, so the two are not independently
swappable.
Needs a Blackwell-class NVIDIA GPU and roughly 28 GB of weights in total.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-nvfp4-vllm-cpp
Qwen3.6-35B-A3B in NVFP4, served by vllm.cpp. A 35B mixture-of-experts model
with roughly 3B parameters active per token, so it reads like a much larger
model while costing about as much per token as a small one.
This is the engine's gated MoE checkpoint: token-for-token identical to vLLM
over the 315-prompt battery on both the synchronous and asynchronous paths,
at 0.92x to 0.97x vLLM's throughput from concurrency 1 to 32.
The architecture is a gated-delta-net hybrid, so automatic prefix caching is
off by default here where it would be on for a dense model. That is the
engine's own default and this entry does not override it.
Needs a Blackwell-class NVIDIA GPU and roughly 23 GB of weights plus KV cache.
Repository: localaiLicense: apache-2.0
qwen3.6-35b-a3b-nvfp4-mtp-vllm-cpp
Qwen3.6-35B-A3B NVFP4 on vllm.cpp with MTP speculative decoding enabled.
The draft head ships inside the checkpoint's own mtp.* tensors, so there is
no second model to download. On this model the speculative path is
token-exact against speculation-off on both the synchronous and asynchronous
schedulers.
Same weights as qwen3.6-35b-a3b-nvfp4-vllm-cpp; install that entry instead if
you would rather not spend the extra memory on speculative state. MTP is
depth 1 by construction on this engine.
Repository: localaiLicense: apache-2.0
qwen3-coder-30b-a3b-vllm-cpp
Qwen3-Coder-30B-A3B on vllm.cpp: a coding and agentic-tool-use model, 30B
total parameters with about 3B active per token, gated token-exact against
vLLM on this engine.
The tool-call parser is named explicitly rather than auto-detected, and that
matters here. Qwen3-Coder's tool dialect is byte-identical on the wire to
another family's, so template sniffing cannot separate the two and would fall
back to the wrong parser. With qwen3_coder named, tool calls arrive as real
tool_calls on the OpenAI response.
This is the bf16 checkpoint, roughly 57 GB of weights, which is what the
engine was gated on. Being bf16 rather than NVFP4 it does not need Blackwell
on its own account, but LocalAI's CUDA images for this backend are currently
built for Blackwell-family GPUs only, so on an older card use the CPU build.
Repository: localaiLicense: apache-2.0
minimax-h3-fl2va-q4
MiniMax-H3 served by vllm.cpp, LocalAI's own C++ port of vLLM. It generates
video AND audio jointly from a text prompt, so a clip comes back as an MP4
with a real soundtrack rather than a silent render: ask for speech in the
prompt and the model lip-syncs it.
This is the Q4_K_M quantisation of the FL2VA partition, which serves
text-to-video (t2va) and first/last-frame conditioning (fl2va). Reference
conditioning (ref2va) is a different checkpoint and is refused by this one.
Roughly 40 GB of weights across five files, plus the two VAE configs that
carry the latent statistics. The default canvas is 1344x768
at 124 frames and 24 fps, about 5.2 seconds. Generation is slow — measured
at roughly 176 s per denoise step at that canvas on a 20-SM device, so the
50-step default is a multi-hour job. Muxing the finished frames needs
ffmpeg on the host.
Repository: localaiLicense: other
minimax-h3-ref2va-q4
MiniMax-H3 served by vllm.cpp, LocalAI's own C++ port of vLLM. It generates
video AND audio jointly, so a clip comes back as an MP4 with a real
soundtrack rather than a silent render.
This is the Q4_K_M quantisation of the Ref2VA partition, the one that takes
REFERENCE conditioning: a reference image, a reference clip, or reference
audio, prepended as their own blocks so the subject or style carries into the
generated video. For plain text-to-video or first/last-frame conditioning use
minimax-h3-fl2va-q4 instead - the two partitions are separate checkpoints and
each refuses the other's tasks.
Use this Q4_K_M build, NOT the NVFP4 Ref2VA weights: NVFP4 renders a
multicolour patch grid, and it took three investigations upstream to
establish that the fault is the quantisation rather than the reference path.
On Q4_K_M the same code renders coherently.
Roughly 40 GB of weights across five files, plus the two VAE configs that
carry the latent statistics. The default canvas is 1344x768 at 124 frames and
24 fps, about 5.2 seconds. Generation is slow - roughly 176 s per denoise
step at that canvas on a 20-SM device, so the 50-step default is a multi-hour
job. Muxing the finished frames needs ffmpeg on the host.
Repository: localaiLicense: other
qwen3-4b-thinking-2507-gspo-easy
**Model Name:** Qwen3-4B-Thinking-2507-GSPO-Easy
**Base Model:** Qwen3-4B (by Alibaba Cloud)
**Fine-tuned With:** GRPO (Generalized Reward Policy Optimization)
**Framework:** Hugging Face TRL (Transformers Reinforcement Learning)
**License:** [MIT](https://huggingface.co/leonMW/Qwen3-4B-Thinking-2507-GSPO-Easy/blob/main/LICENSE)
---
### 📌 Description:
A fine-tuned 4-billion-parameter version of **Qwen3-4B**, optimized for **step-by-step reasoning and complex problem-solving** using **GRPO**, a reinforcement learning method designed to enhance mathematical and logical reasoning in language models.
This model excels in tasks requiring **structured thinking**, such as solving math problems, logical puzzles, and multi-step reasoning, making it ideal for applications in education, AI assistants, and reasoning benchmarks.
### 🔧 Key Features:
- Trained with **TRL 0.23.1** and **Transformers 4.57.1**
- Optimized for **high-quality reasoning output**
- Part of the **Qwen3-4B-Thinking** series, designed to simulate human-like thought processes
- Compatible with Hugging Face `transformers` and `pipeline` API
### 📚 Use Case:
Perfect for applications demanding **deep reasoning**, such as:
- AI tutoring systems
- Advanced chatbots with explanation capabilities
- Automated problem-solving in STEM domains
### 📌 Quick Start (Python):
```python
from transformers import pipeline
question = "If you had a time machine, but could only go to the past or the future once and never return, which would you choose and why?"
generator = pipeline("text-generation", model="leonMW/Qwen3-4B-Thinking-2507-GSPO-Easy", device="cuda")
output = generator([{"role": "user", "content": question}], max_new_tokens=128, return_full_text=False)[0]
print(output["generated_text"])
```
> ✅ **Note**: This is the **original, non-quantized base model**. Quantized versions (e.g., GGUF) are available separately under the same repository for efficient inference on consumer hardware.
---
🔗 **Model Page:** [https://huggingface.co/leonMW/Qwen3-4B-Thinking-2507-GSPO-Easy](https://huggingface.co/leonMW/Qwen3-4B-Thinking-2507-GSPO-Easy)
📝 **Training Details & Visualizations:** [WandB Dashboard](https://wandb.ai/leonwenderoth-tu-darmstadt/huggingface/runs/t42skrc7)
---
*Fine-tuned using GRPO — a method proven to boost mathematical reasoning in open language models. Cite: Shao et al., 2024 (arXiv:2402.03300)*
Repository: localaiLicense: apache-2.0