Hunyuan A13B MoE in Practice: 80B Total Parameters, 64 Experts, and the Deployment Trade-offs
Hunyuan A13B MoE in Practice: 80B Total Parameters, 64 Experts, and the Deployment Trade-offs
Hunyuan-A13B is the only Hunyuan model that publishes a Mixture-of-Experts architecture: its official configuration lists 80.39 billion total parameters, 64 experts, and 32 layers, and the name encodes that each token activates roughly a 13B-scale slice of them. That structure produces two apparently contradictory properties at once: inference compute stays close to a 13B dense model, while the weight total is an 80B-class affair, and all of it has to reach memory. This article covers the structure, the memory math, what the official GGUF is really for, and how A13B splits work with hy4.
The 30-second version
A13B is a server-class model, not a workstation model: the official Q4_K_M GGUF file is 48.8GB and Q4_0 is 45.4GB, so a single consumer GPU cannot hold it. You need multiple GPUs, CPU offload, or a machine with large unified memory. hy4 is the model that runs on one machine. If you want Hunyuan locally, start at hy4 rather than A13B.
Naming and structure first
| Item | Value (from the official config) |
|---|---|
| Architecture | HunYuanMoEV1ForCausalLM |
| Total parameters | 80,393,183,232 (about 80.39 billion) |
| Experts | 64 |
| Layers | 32 |
| Hidden size | 4096 |
| Attention heads / KV heads | 32 / 8 (grouped-query attention) |
| Expert intermediate size | 3072 |
| Maximum position length | 32768 |
| Vocabulary size | 128,167 |
The A in A13B means Active
Tencent uses a naming convention where the leading number is the parameter mass activated per token (13B), while the total parameter count is nowhere in the name. A13B's real total is 80.39 billion. That gap is the root of every deployment decision: compute scales with 13B, storage scales with 80B.
The mechanism: each layer's MLP is split into multiple experts, and a router selects a few of them per token while the rest sit idle. The consequences:
- Compute cost approaches a 13B dense model, because only a few experts run per token.
- Weight storage stays at the 80B level, because every expert still has to be loaded.
- Batching amortizes the cost: across a batch, different tokens activate different experts, so the GPU's compute units fill more completely.
MoE saves compute, not memory
This is the most common misjudgement. People see "13B active" and assume it fits in 13B-class memory. In reality you are paying for 80B weights: around 40GB even at 4-bit, and the official Q4_K_M GGUF measures 48.8GB on disk once quantization metadata and higher-precision layers are included. When memory runs short, the engine offloads layers to system RAM and throughput collapses by an order of magnitude, which is not "a bit slower" but "practically unusable".
The memory ledger: three layers to count
First: weights
| Weight format | Size (measured from the official GGUF) |
|---|---|
| BF16 original | about 161GB |
| Official FP8 repository | about 80GB |
| Official GPTQ-Int4 repository | on the order of 20GB |
| GGUF Q4_0 | 45.4GB |
| GGUF Q4_K_M | 48.8GB |
| GGUF Q8_0 (two shards) | 47.5GB + 38.0GB = about 85.5GB |
Why the official GGUF is worth the download
For an 80B model, the official GGUF matters because of sharding and CPU offload usability: Q8_0 ships as two shards that load piecewise, and Q4_K_M gives a more even quality loss than Q4_0. Community conversions usually ship one huge file with less friendly sharding and metadata handling, which makes offloading harder exactly when you need it. Publishing an official GGUF as a first-class artifact is rare among domestic open models.
Second: the KV cache
KV cache ~= 2 x 32 layers x 8 KV heads x 128 head dim x context length x 2 bytes
~= 128KB x context length
A full 32K window costs about 4GB, and an 8K context about 1GB. For A13B this is not the dominant term — weights are — but it multiplies under concurrency: concurrent requests times context length determines the total cache footprint.
Third: runtime
The CUDA context, multi-GPU communication buffers, and expert-parallel intermediate tensors. Under multi-GPU this item grows noticeably.
Hardware reality: three workable shapes
| Shape | Configuration | Notes |
|---|---|---|
| Multi-GPU server | 2x48GB or 4x24GB, vLLM tensor parallelism | The orthodox setup, where official FP8 weights fit best |
| Single GPU with CPU offload | 1x24GB plus large RAM, llama.cpp layer offload | It runs, but the speed trade is brutal; useful for validation |
| Hosted inference | Use a service that already serves A13B | No cards of your own, no expert parallelism to manage |
Engineering advice
If your goal is a usable Hunyuan on your own machine, start at hy4. If your goal is to measure the A13B capability ceiling, use multiple GPUs or a hosted service and treat A13B as a reference generator: produce evaluation sets, produce reference answers, measure how far smaller models fall short. Do not try to squeeze it onto a single workstation GPU.
Deployment notes
Engine version: an MoE architecture needs llama.cpp or vLLM with Hunyuan expert support. If loading fails, upgrade the engine first.
Parallelism: use tensor parallelism in vLLM to shard the weights. Expert parallelism rarely pays off on consumer hardware.
Quantization choice:
| Your situation | Choice |
|---|---|
| Multi-GPU server, balancing speed and quality | Official FP8 |
| Tight on memory but quality matters | Official GPTQ-Int4 |
| CPU offload or llama.cpp ecosystem | Official GGUF Q4_K_M |
| Building an accuracy baseline | Official GGUF Q8_0 (mind the two shards) |
Batching: one real benefit of MoE is concurrent throughput. Batch requests fill the GPU more completely than serial single requests, which is exactly why it suits a server role rather than a local desktop.
What to measure yourself: the numbers that decide a deployment are peak memory, time to first token at your real context length, and sustained tokens per second at your real concurrency. Measure them on your own hardware with your own prompt set, because expert routing behaviour, PCIe topology, and the engine build all move these numbers more than any published benchmark does.
How it splits work with hy4
| Dimension | A13B | hy4 |
|---|---|---|
| Structure | MoE, 80.39B total parameters across 64 experts | Dense, 4.22B parameters |
| Weight size at 4-bit | 48.8GB | 2.61GB |
| Native maximum position length | 32768 | 262144 |
| KV cache per token | about 128KB | about 144KB |
| Hardware floor | Server-class multi-GPU | A single GPU, even CPU-only |
| Role | Capability ceiling, reference generation, complex work | Single machine, edge, high-concurrency light tasks |
A practical pairing
Use A13B offline to generate a batch of high-quality question-answer pairs and reference summaries, then let hy4 handle classification, rewriting, and routing on the live path. The two do not conflict: the large model defines the standard, the small model carries the traffic.
FAQ
How many parameters does A13B actually have?
Can A13B fit on a consumer GPU?
Does Hunyuan ship an official GGUF?
Is an MoE model faster than a dense model of the same size?
Is A13B's context shorter than hy4's?
Can I call Hunyuan through an API?
Related reading
- hy4 model page — specification, quantization matrix and memory math
- hy4-preview channel notes — what the 1M context channel is for
- Tencent Hunyuan family overview — every Hunyuan product line in one place
- Hunyuan hy4 and A13B local deployment — deployment detail for the two main text lines
- hy4 GGUF local deployment — the full path for the model that fits on one machine
- hy4 model selection guide — sizes and where A13B fits
- VRAM calculation guide — the general memory math
- Qwen vs Hunyuan — ecosystem activity versus quantization completeness
Sources
Structure and file sizes follow the official repository config and file listing, sampled in November 2026. Treat throughput figures as something to measure on your own hardware.