EN ID MS TH TL VI ZH

hy4 Ollama Integration: Wiring a Local Hunyuan into Dify, Cursor, and Any OpenAI-Compatible Client

Once hy4 runs locally, the daily question is whether your tools can call it. This guide covers importing a local GGUF into Ollama with a Modelfile, serving it, connecting Dify, Cursor, Open WebUI, and editor plugins, running local and cloud models side by side, and the failure modes that trip people up.

hy4 Ollama Integration: Wiring a Local Hunyuan into Dify, Cursor, and Any OpenAI-Compatible Client

Getting hy4 running solves only half the problem. What actually shapes your daily experience is whether your tools can call it. Ollama is strong on exactly this point: it ships its own model management and an OpenAI-compatible endpoint, and tools such as Dify, Cursor, Open WebUI, and Continue already understand it without any code changes. This article walks the whole chain: importing a local GGUF, serving it, configuring clients, running local and cloud side by side, and troubleshooting.

The 30-second version

Import a local GGUF into Ollama with ollama create and a Modelfile, serve it on port 11434, then point clients at http://127.0.0.1:11434/v1 with any placeholder key, and validate with the model list endpoint. Dify, Cursor, and Open WebUI all speak OpenAI-compatible protocols, so this is configuration work rather than integration work.

Step one: prepare the GGUF weights

bash

mkdir -p ~/models && cd ~/models
huggingface-cli download bartowski/tencent_Hunyuan-4B-Instruct-GGUF \
  Q4_K_M.gguf --local-dir .
ls -lh

Why a community GGUF rather than the official quantizations

The official releases are FP8, GPTQ-Int4, and AWQ-Int4, which need a serving stack such as vLLM. Ollama and llama.cpp work with GGUF, so a community conversion is the shortest path for local deployment. bartowski publishes the widest tier coverage with the highest download counts, which makes it the safe default.

Step two: create the Ollama model

bash

cat > ~/models/Modelfile <<'EOF'
FROM ./Q4_K_M.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

PARAMETER num_ctx 8192
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER stop "<|im_end|>"
EOF

cd ~/models && ollama create hy4-local -f Modelfile
ollama list

Copy the TEMPLATE from the model card

This is the most devious failure in local deployment: the model loads, it produces tokens, and the answers are terrible, which looks like a weak model. The cause is almost always a mismatched chat template, and Hunyuan templates are not identical across generations. Copy the template from the Usage section of the matching model card. Include the stop parameter too, otherwise the model never stops.

Step three: serve and verify

bash

# run in the foreground, then use another terminal for the checks
ollama serve

# native API
curl http://127.0.0.1:11434/api/chat -d '{
  "model": "hy4-local",
  "messages": [{"role": "user", "content": "用一句话解释向量数据库"}],
  "stream": false
}'

Verify the OpenAI-compatible surface:

bash

# model list
curl http://127.0.0.1:11434/v1/models

# chat
curl http://127.0.0.1:11434/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "hy4-local",
  "messages": [{"role": "user", "content": "写一个 Python 函数解析 ISO 8601 时间"}]
}'

Two APIs, two jobs

/api/chat is the native Ollama endpoint and supports extensions such as think and images. /v1/* is the OpenAI-compatible surface that third-party tools use. Stick to /v1 when wiring tools, and only reach for the native API when you need Ollama-specific features.

Step four: connect an OpenAI-compatible client

python

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:11434/v1",
    api_key="ollama",           # placeholder, Ollama does not validate it
)
resp = client.chat.completions.create(
    model="hy4-local",
    messages=[{"role": "user", "content": "你好"}],
)
print(resp.choices[0].message.content)

Step five: wire up specific tools

Dify

In Settings, Model Providers, OpenAI-API-compatible, set the API Base URL to http://127.0.0.1:11434/v1, the API Key to ollama, and the model name to hy4-local. When Dify runs in Docker, 127.0.0.1 refers to the container itself, so use the host IP or service name instead.

Cursor

In Cursor settings under Models, add a model through the OpenAI provider, point the Base URL at the local address, and use hy4-local as the model name. Cursor support for local endpoints varies by version, and manually entering the model name works when automatic discovery fails.

Open WebUI

In Admin Settings, Connections, OpenAI, enter the same address, then enable hy4-local under Models. Note that a containerised Open WebUI must reach the host through host.docker.internal rather than 127.0.0.1.

Editor plugins such as Continue and Cline

Pick the OpenAI provider in the plugin model settings and fill in the local base URL and model name. Most of these plugins have a dedicated local models or custom provider option, which is cleaner than editing global settings.

Docker networking is the most common trap

Inside a container, localhost points at the container itself. Two fixes: add extra_hosts: ["host.docker.internal:host-gateway"] to the compose file and use host.docker.internal:11434, or have Ollama listen on all interfaces with OLLAMA_HOST=0.0.0.0 ollama serve. The second option exposes your model to the whole local network, so add access control before doing it.

Step six: local and cloud side by side

Many teams run a local hy4 and cloud models at the same time. Give different tools different endpoints, or route inside one client:

python

import os

def pick_client(task: str):
    if task in {"classify", "rewrite", "summarize"}:
        return OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="ollama")
    return OpenAI(base_url="https://exaiv.com/api/ai/v1", api_key=os.environ["EXAIV_KEY"])

Do not mix up model names

The local model name is hy4-local, which you chose, while a cloud model ID is something that genuinely exists in the catalog, such as kimi-k3. A wrong model name usually fails with a model-not-found error rather than a connection error, so check the name first and the network second.

Troubleshooting

Symptom	Cause	Fix
Garbled or irrelevant output	Wrong chat template	Fix TEMPLATE and stop per the model card
Connection refused	Service not running, or not on 11434	Start `ollama serve` and verify the port with curl
Container cannot connect	Docker network isolation	Use host.docker.internal or bind 0.0.0.0
Answer cut off mid-sentence	Context or max_tokens too small	Raise num_ctx and max_tokens
Model name error	Cloud ID or a typo	Check `ollama list` for the real name
Very slow first response	Model loading into memory	Expected, later requests are fast
Out of memory	num_ctx set too high	Lower the context, since the KV cache scales linearly

FAQ

Should I use Ollama or llama.cpp?

For maximum performance and tunable parameters, llama.cpp. For ecosystem compatibility, since Open WebUI, Dify, and Cursor work out of the box, Ollama. Both expose an OpenAI-compatible endpoint and can share the same GGUF files.

Where do I get the TEMPLATE for the Modelfile?

From the Usage section of the matching model card. Hunyuan templates differ across generations, and a wrong template still produces fluent tokens with poor answers, which is the most common misdiagnosis in local deployment.

How does Dify in Docker reach Ollama on the host?

Inside the container, 127.0.0.1 refers to the container. Add extra_hosts mapping host.docker.internal to the host gateway in the compose file, then use that hostname with port 11434 as the Base URL.

Can I expose Ollama as a service?

Yes, with OLLAMA_HOST=0.0.0.0 ollama serve it listens on every interface. Ollama has no authentication by default, so anyone on the network can call your model. Put a reverse proxy and access control in front before exposing it publicly.

Can I call Hunyuan through the EXAIV cloud?

The EXAIV cloud catalog does not carry Hunyuan today; check the live model directory. Hunyuan goes through the self-hosted path, and this article is about wiring that into your own toolchain.

hy4 GGUF local deployment — quantization tiers, memory math, and llama.cpp commands
Open WebUI with a local Hunyuan — the full self-hosted chat interface setup
hy4 local RAG in practice — pointing a knowledge base at a local endpoint
Dify integration — connecting cloud models to Dify
Open WebUI integration — connecting cloud models to a self-hosted interface
hy4 model selection guide — sizes and where A13B fits

Sources

Repository lists and tiers move as the community updates; this article reflects the state at sampling time.

One key for Chinese open models

EXAIV serves Qwen, DeepSeek, Kimi and GLM through one OpenAI-compatible endpoint. Sign up and start calling

See credit packs

← Back to Integrations

EN ID MS TH TL VI ZH

hy4 Ollama Integration: Wiring a Local Hunyuan into Dify, Cursor, and Any OpenAI-Compatible Client

The 30-second version

Step one: prepare the GGUF weights

bash

mkdir -p ~/models && cd ~/models
huggingface-cli download bartowski/tencent_Hunyuan-4B-Instruct-GGUF \
  Q4_K_M.gguf --local-dir .
ls -lh

Why a community GGUF rather than the official quantizations

Step two: create the Ollama model

bash

cat > ~/models/Modelfile <<'EOF'
FROM ./Q4_K_M.gguf

TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .Prompt }}<|im_start|>user
{{ .Prompt }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

PARAMETER num_ctx 8192
PARAMETER temperature 0.7
PARAMETER top_p 0.8
PARAMETER stop "<|im_end|>"
EOF

cd ~/models && ollama create hy4-local -f Modelfile
ollama list

Copy the TEMPLATE from the model card

Step three: serve and verify

bash

# run in the foreground, then use another terminal for the checks
ollama serve

# native API
curl http://127.0.0.1:11434/api/chat -d '{
  "model": "hy4-local",
  "messages": [{"role": "user", "content": "用一句话解释向量数据库"}],
  "stream": false
}'

Verify the OpenAI-compatible surface:

bash

# model list
curl http://127.0.0.1:11434/v1/models

# chat
curl http://127.0.0.1:11434/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "hy4-local",
  "messages": [{"role": "user", "content": "写一个 Python 函数解析 ISO 8601 时间"}]
}'

Two APIs, two jobs

Step four: connect an OpenAI-compatible client

python

from openai import OpenAI

client = OpenAI(
    base_url="http://127.0.0.1:11434/v1",
    api_key="ollama",           # placeholder, Ollama does not validate it
)
resp = client.chat.completions.create(
    model="hy4-local",
    messages=[{"role": "user", "content": "你好"}],
)
print(resp.choices[0].message.content)

Step five: wire up specific tools

Dify

Cursor

Open WebUI

Editor plugins such as Continue and Cline

Docker networking is the most common trap

Step six: local and cloud side by side

Many teams run a local hy4 and cloud models at the same time. Give different tools different endpoints, or route inside one client:

python

import os

def pick_client(task: str):
    if task in {"classify", "rewrite", "summarize"}:
        return OpenAI(base_url="http://127.0.0.1:11434/v1", api_key="ollama")
    return OpenAI(base_url="https://exaiv.com/api/ai/v1", api_key=os.environ["EXAIV_KEY"])

Do not mix up model names

Troubleshooting

Symptom	Cause	Fix
Garbled or irrelevant output	Wrong chat template	Fix TEMPLATE and stop per the model card
Connection refused	Service not running, or not on 11434	Start `ollama serve` and verify the port with curl
Container cannot connect	Docker network isolation	Use host.docker.internal or bind 0.0.0.0
Answer cut off mid-sentence	Context or max_tokens too small	Raise num_ctx and max_tokens
Model name error	Cloud ID or a typo	Check `ollama list` for the real name
Very slow first response	Model loading into memory	Expected, later requests are fast
Out of memory	num_ctx set too high	Lower the context, since the KV cache scales linearly

FAQ

Should I use Ollama or llama.cpp?

Where do I get the TEMPLATE for the Modelfile?

How does Dify in Docker reach Ollama on the host?

Inside the container, 127.0.0.1 refers to the container. Add extra_hosts mapping host.docker.internal to the host gateway in the compose file, then use that hostname with port 11434 as the Base URL.

Can I expose Ollama as a service?

Can I call Hunyuan through the EXAIV cloud?

The EXAIV cloud catalog does not carry Hunyuan today; check the live model directory. Hunyuan goes through the self-hosted path, and this article is about wiring that into your own toolchain.

hy4 GGUF local deployment — quantization tiers, memory math, and llama.cpp commands
Open WebUI with a local Hunyuan — the full self-hosted chat interface setup
hy4 local RAG in practice — pointing a knowledge base at a local endpoint
Dify integration — connecting cloud models to Dify
Open WebUI integration — connecting cloud models to a self-hosted interface
hy4 model selection guide — sizes and where A13B fits

Sources

Repository lists and tiers move as the community updates; this article reflects the state at sampling time.

One key for Chinese open models

EXAIV serves Qwen, DeepSeek, Kimi and GLM through one OpenAI-compatible endpoint. Sign up and start calling

See credit packs

← Back to Integrations