Tencent Hunyuan hy4 and A13B Local Deployment: The Hunyuan-4B Lightweight Line and the A13B MoE Flagship
Tencent Hunyuan hy4 and A13B Local Deployment
Tencent's open text models have an easily-missed detail: they are not under the Tencent-Hunyuan organization (which hosts the HunyuanDiT image models) but under tencent. This text line has two mainstays: Hunyuan-4B (community shorthand hy4) as the lightweight edge line, and Hunyuan-A13B-Instruct as the MoE flagship with 68K downloads. Both lines have official FP8/GPTQ-Int4/AWQ-Int4 quantizations, and community GGUF conversions exist. This article covers their positioning, deployment, and how they compare to Qwen and ERNIE lightweight lines.
The short version
hy4 = Hunyuan-4B-Instruct, the lightweight edge line: 4B parameters with FP8/GPTQ/AWQ and community GGUF. A13B = Hunyuan-A13B-Instruct, the MoE flagship (13B active) at 68K downloads. Locally: 4B goes straight to Ollama via GGUF; A13B runs on llama.cpp or vLLM (MoE needs an MoE-capable engine).
A note on the Hunyuan DiT confusion
If you search HuggingFace for "Hunyuan", the first results are likely HunyuanDiT image-generation models under Tencent-Hunyuan — ControlNet variants, distilled pipelines, and TensorRT builds from 2024. These are a completely different product line from the text LLMs.
The text LLMs (Hunyuan-4B, Hunyuan-7B, Hunyuan-A13B) live under the lowercase tencent organization. This naming split trips up a lot of searches, so it is worth bookmarking the right org:
tencent— text LLMs: Hunyuan-4B-Instruct, Hunyuan-7B-Instruct, Hunyuan-A13B-Instruct, plus Pretrain basesTencent-Hunyuan— image/video generation: HunyuanDiT, ControlNets, captioners, IP-Adapter
If a GGUF conversion repo references Tencent-Hunyuan, it is almost certainly a DiT image model, not a chat model — check before downloading.
Fine-tuning from the Pretrain bases
Both lines ship Pretrain versions — Hunyuan-4B-Pretrain and Hunyuan-A13B-Pretrain — for teams that want to fine-tune rather than use the instruct weights directly.
A few practical points:
- Pretrain weights have no chat training: they will not follow instructions well out of the box; that is expected, not a bug
- Start from Pretrain for domain adaptation: if you are training on your own corpus (legal, medical, internal docs), begin from Pretrain and apply your own SFT stage
- A13B MoE fine-tuning is heavier: MoE routing adds complexity to training; budget more VRAM and verify your trainer supports MoE layers
- 4B fine-tuning is accessible: a single consumer GPU can typically handle LoRA on a 4B dense model
If your goal is prompt engineering rather than weight changes, skip Pretrain entirely and use the Instruct weights.
Memory planning for the two lines
The two lines have very different memory profiles, and planning them correctly avoids most local-deployment failures:
Hunyuan-4B (dense): all parameters are active at inference, so memory demand is stable and predictable. An Int4 quantization typically fits in 4-6GB, and the KV cache at 8K context is small. This is the easiest line to plan for.
Hunyuan-A13B (MoE): memory has two components — the full weights (which must be stored even though only 13B are active per token) and the KV cache. Plan for total-weight storage plus KV cache headroom, not just the active size. At Int4, weight storage is manageable on a single larger GPU; at FP8 or BF16, you are into multi-GPU territory.
A practical sequence for a new machine: start with hy4 at Int4 to validate the pipeline, then move to A13B GGUF (Q4_K_M) when you need more capability, keeping n_ctx modest until you have measured real VRAM headroom.
Verify before you trust a GGUF
Because Hunyuan's community GGUF comes from several independent converters, a two-minute verification step saves hours of debugging:
- Check the conversion date on the model card — architecture support in llama.cpp evolves quickly, and an old conversion may fail on a new engine or vice versa
- Load with a short context first (
-c 2048) and ask one factual question; garbled output means the architecture layer is not supported - Only then raise context and GPU layers to your production settings
For the 4B line this rarely matters (dense architectures are broadly supported), but for A13B MoE the routing implementation is engine-version-sensitive — verify before committing to a long download.
The two lines' positioning
| Model | Architecture | Positioning | Downloads |
|---|---|---|---|
| Hunyuan-4B-Instruct | Dense 4B | Lightweight edge | ~1K |
| Hunyuan-4B-Pretrain | Dense 4B | Fine-tune base | 628 |
| Hunyuan-A13B-Instruct | MoE 13B active | Flagship general | 68K |
| Hunyuan-A13B-Pretrain | MoE 13B active | Fine-tune base | 491 |
What A13B means
Same naming scheme as Qwen's A3B/A95B: A stands for Active, and 13B means roughly 13 billion parameters activated per token. Hunyuan-A13B is an MoE with larger total parameters but only 13B active per token, so inference overhead is close to a 13B dense model.
Hunyuan-4B: the lightweight edge line
hy4 is a 4B-parameter dense model with four official variants:
| Variant | Notes |
|---|---|
Hunyuan-4B-Instruct | BF16 raw weights |
Hunyuan-4B-Instruct-FP8 | 8-bit float, half the size |
Hunyuan-4B-Instruct-GPTQ-Int4 | 4-bit weights, even less VRAM |
Hunyuan-4B-Instruct-AWQ-Int4 | AWQ 4-bit, a different quantization algorithm |
At 4B scale, an Int4 quantization typically runs in 4-6GB of VRAM or pure CPU memory, making it a candidate for Raspberry-Pi-class devices, older laptops, or embedded scenarios. Community GGUF (like gabriellarson/Hunyuan-4B-Instruct-GGUF and Disya/Hunyuan-4B-Instruct-Q8_0-GGUF) plugs straight into Ollama/llama.cpp.
Hunyuan-A13B: the MoE flagship line
A13B-Instruct is Hunyuan's open flagship, and 68K downloads indicate real production usage. Three official variants exist:
| Variant | Notes |
|---|---|
Hunyuan-A13B-Instruct | BF16 raw weights |
Hunyuan-A13B-Instruct-FP8 | FP8 quantization, server-friendly |
Hunyuan-A13B-Instruct-GPTQ-Int4 | 4-bit weights for tight VRAM |
The deployment caveat for MoE models: the inference engine must support MoE routing. Recent llama.cpp versions support the Hunyuan MoE architecture, and vLLM supports it too. Before running A13B on Ollama, confirm its underlying llama.cpp version supports the architecture, or it will fail to load or output garbage.
For community GGUF, repos like bullerwins/Hunyuan-A13B-Instruct-GGUF offer multi-tier conversions; Apple Silicon users also have mlx-community/Hunyuan-A13B-Instruct-4bit in MLX format, which runs well on M-series chips.
Deployment command quick reference
Hunyuan-4B via Ollama:
# After downloading a community GGUF, write a Modelfile
# FROM ./models/Hunyuan-4B-Instruct-Q4_K_M.gguf
ollama create hy4 -f Modelfile
ollama run hy4 "Explain what a KV cache is in one sentence"
A13B via llama.cpp:
./build/bin/llama-cli \
-m ./models/Hunyuan-A13B-Instruct-Q4_K_M.gguf \
-ngl 40 \
-p "Explain what a KV cache is in one sentence"
Verify GGUF conversion versions
Hunyuan's community GGUF comes from multiple converters, and quality plus architecture support evolve with llama.cpp versions. Check the conversion date and notes on the model card before downloading; if loading fails, update llama.cpp first and retry.
How it compares to Qwen and ERNIE lightweight lines
Placing Hunyuan in the Chinese lightweight-model coordinate system:
| Model | Params | Quantization variants | Notable trait |
|---|---|---|---|
| Hunyuan-4B | 4B dense | FP8/GPTQ/AWQ/GGUF | Most complete official quantization at 4B scale |
| Qwen3.8-Flash-Next | Lightweight flagship | Official FP8 + community GGUF | Largest ecosystem |
| ERNIE-4.5-0.3B | 0.3B dense | Paddle native | Extreme edge |
Hunyuan-4B's differentiation is the most complete official quantization set — FP8, GPTQ, and AWQ all officially released, which suits teams that need a specific quantization format to plug into their own inference stack. Qwen has a bigger ecosystem, but "four official precisions to choose from" is Hunyuan's real advantage.
Cloud catalogs are the source of truth
At the time of writing, EXAIV's cloud catalog does not carry Hunyuan (current domestic offerings are Qwen/Kimi/GLM; check the live catalog). To run Hunyuan locally use community GGUF or official quantizations; do not use HF weight names as API IDs.
Selection quick reference
| Your scenario | Recommended Hunyuan |
|---|---|
| Edge / embedded / older devices | Hunyuan-4B-Instruct (GPTQ-Int4 or GGUF) |
| Own stack needs a specific quantization format | FP8/GPTQ/AWQ official variants of 4B or A13B |
| Private general chat (ample resources) | Hunyuan-A13B-Instruct (FP8 or GGUF) |
| Apple Silicon local run | mlx-community MLX 4-bit builds |
| Fine-tuning yourself | Pretrain versions (4B-Pretrain / A13B-Pretrain) |
FAQ
What is hy4?
Which HF organization hosts Hunyuan text models?
Can A13B run on Ollama?
4B or A13B?
Can I call Hunyuan via API?
What are the mlx-community versions?
Related reading
- hy4 model page — specification, quantization matrix and memory math
- hy4-preview channel notes — what the 1M context channel is for
- Tencent Hunyuan family overview — every Hunyuan product line in one place
- hy4 model selection guide — how hy3, hy4, the preview channels and A13B divide the work
- hy4 GGUF local deployment — quantization tiers, memory math and commands
- Hunyuan A13B MoE in practice — the deployment ledger for an 80B MoE
- hy4 model page — specification, quantization matrix and memory math
- hy4 local RAG in practice — wiring a knowledge base to a local model
- Running hy4 with Ollama — connecting it to Dify, Cursor and other tools
- Baidu ERNIE Family Overview — Baidu's open family
- DeepSeek V4 Family Overview — the most-downloaded Chinese family
- Kimi K3 and GLM Integration Guide — two families callable on EXAIV
- Qwen GGUF Local Deployment Primer — local deployment basics
- Qwen Model Architecture — understanding MoE architecture