Llama Open Weights: Llama 3.3, Llama 4 Scout and the Local Deployment Boundary

This page maps llama-family-overview using facts verified from HuggingFace metadata and official repository configuration, sampled in November 2026. It separates local open weights from EXAIV cloud IDs and does not invent benchmarks, prices, or converted repositories.

Verified facts

A HuggingFace repository proves that weights or a model card exists; it does not prove that EXAIV serves the model. Download counts are public 30-day community signals, not quality scores. Licences must be read per repository. Config fields prove architecture, not every capability or request payload.

Deployment reality

Choose a runtime that matches the format, measure memory on your own hardware, and keep the model ID, file name, licence, token counts and latency in a run log. Community GGUF files need a named publisher, a stated conversion source and a clear quantization label. The local deployment angle here is intentional: none of these repositories should be called an EXAIV cloud model unless the live catalog lists the ID.

Availability boundary

The safest wording is local open weights, official model card, and measured behaviour on your machine. Availability and prices follow /models and /pricing. A licence tag such as apache-2.0, gemma, llama3.3 or other is a starting point for review, not legal advice.

Situation

Practical pre-flight checklist

Check the repository source, exact file format, chat template, memory peak, licence, and the request format before publishing or deploying. Do not copy a benchmark or price from a stale article. Run a fixed evaluation set after every change to quantization, adapter, runtime or routing. Keep a rollback file and record the exact model ID.

FAQ

Related reading

• Claude family overview — callable IDs
• GPT family overview — ID families
• Open overseas weights — local weights
• Mistral vs Llama vs Gemma — selection
• pricing — live credit packs

Sources

• HuggingFace model API
• EXAIV live model catalog

Parameter, download and licence data sampled in November 2026 and subject to change.

Reusable local deployment checklist

Scattered tutorials cover single points: how to install Ollama, how to download a GGUF, how to configure a plugin. But deploying a local model you can actually use daily is a full chain — is the hardware enough, which model and tier, which file, how to serve it, how to wire it into tools, how to debug. This playbook walks the chain in seven steps, each with executable commands and stop/switch criteria.

A pitfalls checklist before day one

Most local deployment failures trace to a handful of predictable mistakes. Check them before the first download:

• Disk on the system drive: model caches filling the OS partition is the most common surprise. Move the cache directory early.
• RAM below the model size: if you plan any CPU offload, system RAM must cover the offloaded layers plus the OS.
• PSU headroom ignored: transient spikes during multi-GPU inference trip undersized supplies.
• Engine version drift: MoE architectures change requirements between engine releases; pin a version and update deliberately.
• Context set larger than needed: -c consumes VRAM through KV cache immediately; size it to your real usage.

Ten minutes on this list prevents the most common first-day failures — and none of them are about the model itself.

Keeping the deployment healthy over time

A local model is not a set-and-forget install. Three habits keep it healthy:

• Track disk growth: model caches accumulate old versions; prune deliberately each month.
• Watch VRAM creep: as you raise context or add users, peak usage climbs — re-check nvidia-smi after every configuration change.
• Re-validate after engine updates: llama.cpp and Ollama releases can change speed and behavior; rerun your small eval set after upgrading.

A local deployment treated like a small service stays reliable; one treated like an install-and-forget app degrades silently until the day it fails mid-task.

Step 1: check hardware and compute budget

Confirm three things first; if any fails, adjust the target rather than forcing it:

VRAM math: target parameters × per-tier byte coefficient + KV cache + 2GB overhead. Full formulas and worked examples are in VRAM calculation in practice. One line: 27B wants 24GB-class VRAM, 14B-class fits 16GB, 4B-class is easy on 8GB.

Step 2: work backward to model and tier

With a VRAM budget, reverse-lookup what you can run:

Family selection: Which Qwen Model to Use, DeepSeek V4 Family Overview, Hunyuan hy4 and A13B. Tier tradeoffs: GGUF Quantizer Comparison.

Step 3: download GGUF

Download the target model's GGUF from HuggingFace. Two routes:

# Download a specific file with huggingface-cli (confirm the real filename on the model card)
huggingface-cli download <author>/<model>-GGUF \
  <model>-Q4_K_M.gguf --local-dir ./models

Or use a runtime that pulls directly (some models support an official pull in Ollama). Check the file size on the model card before downloading so you do not discover it does not fit afterward.

Step 4: get it running

Route A: Ollama (fastest to validate)

# Write a Modelfile pointing at the local GGUF
# FROM ./models/<model>-Q4_K_M.gguf
ollama create mymodel -f Modelfile
ollama run mymodel "Introduce yourself in one sentence"

Route B: llama.cpp (control and serving)

git clone https://github.com/ggerganov/llama.cpp && cd llama.cpp && make -j

./build/bin/llama-server \
  -m ./models/<model>-Q4_K_M.gguf \
  -ngl 35 \
  -c 8192 \
  --host 127.0.0.1 --port 8080

Step 5: expose a local endpoint

To wire the model into tools you need a long-running HTTP endpoint:

• Ollama: listens on 127.0.0.1:11434 by default, exposing an OpenAI-compatible API
• llama.cpp: llama-server listens on your chosen port, also compatible with /v1/chat/completions

Step 6: wire it into your tools

With the local endpoint running, integration is "point at the address and pick a model":

One practical detail: editor plugins usually have timeouts, while a local model's first load takes time — raise the timeout, or run a few warmup requests before integrating.

Step 7: monitor, debug, and fall back

Monitoring: nvidia-smi for VRAM peaks; llama.cpp logs for KV cache allocation and tokens/s.

Fixed debug order:

1. VRAM short → lower context (-c) or lower the tier
2. Context fine but slow → reduce RAM-offloaded layers, confirm the GPU is actually used
3. Garbled output or load failure → update the engine (MoE architectures are engine-version sensitive)
4. Tool-side errors → curl the endpoint directly first, to separate endpoint issues from tool config issues

Fallback: local models are the mainstay, cloud is the safety net. When local is slow, down, or needs flagship quality, one fallback call to a cloud API (EXAIV's OpenAI-compatible endpoint, IDs per the live /models catalog) keeps the loop unbroken. For the cost tradeoff, see the local compute cost model.

FAQ

Related reading

• Buying Local Compute: A GPU Purchase Guide — read before buying a card
• VRAM Calculation in Practice — formulas and measurement
• Local Compute Cost Model — buy vs rent vs API
• Qwen GGUF Local Deployment Primer — quantization and deployment basics
• Chinese Open Weights vs US Flagships — the local-versus-cloud picture