Fine-tuning guide: LoRA and QLoRA for Chinese open models

Fine-tuning guide: LoRA and QLoRA for Chinese open models is documented here with numbers verified directly against the HuggingFace API and the raw config.json of the corresponding repository, sampled in November 2026. No benchmark score, price, or repository name is invented; anything the config cannot prove is labelled as needing measurement.

Verified specification

|---|---|---|---|

Reference table

|---|---|---|---|

Reference table

|---|---|

Reference table

|---|---|---|

What the numbers actually confirm

The distinction between verified fact and claim matters most for multimodal wording. A vision_config and an image_placeholder in the official config confirm an image-capable conditional-generation architecture; they do not by themselves prove video generation, a specific request payload, or any benchmark result. The HuggingFace tag image-text-to-text describes the input direction. Always treat the official model card as the final source of truth.

Configuration and troubleshooting

Memory planning follows the same rule for every trillion-parameter MoE: compute scales with active parameters while memory scales with total parameters. A 1T model at 4-bit still needs roughly 500GB for weights alone, and a 2.78T model is larger still, so these are multi-GPU server workloads rather than single consumer cards. For a single card, look at the lightweight families. The local path is vLLM or SGLang with official weights, and community conversions should be checked for source and quantization label first.

Honest boundaries

On EXAIV the live /models catalog is authoritative: only kimi-k3 is callable today, through the OpenAI-compatible endpoint at https://exaiv.com/api/ai/v1. Integration changes only base_url, api_key and model; streaming and tool calling keep the standard OpenAI format. Prices always follow the live /pricing page and are never hard-coded here. Keep a log of model ID, token counts and latency so a later model swap has the same baseline.

Terminology and pre-flight checklist

Total parameters, active parameters and the KV cache are three different quantities. Total parameters are all the weights that must be loaded into memory; active parameters are the subset that computes a single token. KV cache is temporary memory that grows linearly with context length, so a 1M window is a ceiling rather than a free default. Quantization lowers the precision of stored weights to shrink files; it does not reduce the parameter count. A vision_config in a config.json describes architecture, not a wire format you may safely invent.

Before shipping anything built on these notes, run six checks. Re-read the numbers from the live source on the day you publish, because download counters move weekly. Open the live /models catalog and confirm the model ID you plan to call is still there. Open /pricing rather than copying a price into your application. Never hard-code a request body you have not validated against the official reference. Record the model ID, input and output token counts, and latency for a representative set of prompts, then keep that set unchanged so a later model swap produces a comparable reading. Read the licence on the model card before commercial use, since vendor-specific terms are not a permissive default. Add exponential backoff for retryable requests, set a timeout, and log the error class so that a 401, a 404 and a 429 never get the same treatment.

FAQ

Related reading

• Kimi K3 API guide — first API call
• GLM family overview — callable catalog
• hy4 model selection guide — sizing and selection
• Claude Code third-party models — support boundary
• pricing — live credit packs

Sources

• EXAIV live model catalog

Parameter and file data sampled in November 2026 and subject to change.

Common misreads

The most frequent confusion is treating a catalog ID as a HuggingFace weight name: one is for API requests, the other for local downloads, and the two naming systems must not be mixed. A second misread is treating download counts as a quality score when they only reflect community size and the amount of prior discussion available. A third is inferring a wire format directly from config fields, which produces example code that looks complete but fails in production. A minimal verified request is always more reliable than an impressive-looking one. Log the model ID, a summary of the request body, and the response status together: check the key and header first for a 401, re-check the live catalog for a 404, and apply backoff for a 429 before changing any model configuration. That keeps the troubleshooting path stable and reusable across later guides.

Fine-tuning workflow notes

A fine-tuning guide is only useful if it separates what you can verify from what you have to measure. Start with the base weights rather than a quantized file: quantization lowers the precision of stored weights to shrink them, and a 4-bit file is a poor starting point for gradient-based training. Keep the official FP8 or BF16 weights as the training source, then quantize afterwards for serving. LoRA and QLoRA are the two parameter-efficient routes most teams reach for first; the practical difference is how much memory the base weights occupy during training, and the exact answer depends on your hardware, your batch size and your sequence length, so measure it on your own machine instead of copying a number from somebody else's benchmark.

Data preparation decides more of the outcome than the choice of adapter. Build a fixed evaluation set before you touch the training set, keep it unchanged through every experiment, and record the model ID, dataset version, token counts, learning-rate schedule and evaluation score for each run. A 1M context window on a flagship model is a ceiling for input size, not a reason to feed whole repositories into every training example. Trim irrelevant material first, and keep a small held-out set that reflects the prompts you actually expect in production.

Three red lines keep the guide honest. Do not publish a benchmark score you have not reproduced. Do not hard-code a price or a repository name that you have not checked on the official model card. Do not describe a third-party gateway workflow as an officially supported feature of the upstream vendor. When you swap an adapter, a base model or a quantization tier, rerun the same evaluation set and log the delta. That single habit turns fine-tuning from a collection of anecdotes into an experiment you can repeat, and it gives every later article on the same topic a comparable baseline.

Serving the adapted model

After training, the serving path is a separate decision. Export the adapter or merge it into the base weights, then choose a serving framework that matches the weight format: vLLM or SGLang for official FP8 or BF16 checkpoints, and llama.cpp or Ollama for community GGUF conversions. A community conversion should always be checked for its publisher, its stated conversion source and its quantization label before it reaches production. Two files both named Q4_K_M from different publishers are not interchangeable. If a model loads but emits garbled output, suspect the chat template before blaming the weights, and store the template next to the weights so the pair can be audited together.

Cost is the last constraint. Calls are billed by input and output tokens, not by request count, and a long system prompt repeated on every call is the fastest way to inflate spending. Cache stable prefixes where the platform supports it, stream only where users need to see output immediately, and keep the live /pricing page as the only source of numbers. Availability and IDs belong to the live /models catalog. Together with the evaluation log from the training phase, these records let you attribute a regression to a data change, an adapter change or a routing change instead of guessing.