Agent Framework in Practice: Building Multi-Step Workflows with Open Chinese Models

Single-turn Q&A is only the starting point for LLM applications. Real value comes from having the model execute multi-step operations: query data → analyze results → call an API → generate a report. That is an agent — a loop built around an LLM where the model decides "what to do next" and the framework executes and feeds results back. This guide builds a runnable agent workflow with LangGraph and open Chinese models.

The agent core loop

An agent's minimal working unit is a loop:

User input → LLM analyzes → decides to call tool → executes tool → results fed back → LLM continues → … → final answer

The key is that the model decides what to do next (not code-hardcoded steps), while the framework "executes the model's decision and feeds results back."

LangGraph minimal implementation

LangGraph uses a StateGraph to manage agent flow. Here is a minimal agent with tool calling:

# pip install langgraph langchain-openai

from langgraph.graph import StateGraph, MessagesState, END
from langchain_core.messages import HumanMessage, SystemMessage
from langchain_openai import ChatOpenAI

# Connect to an open Chinese model via EXAIV
llm = ChatOpenAI(
    model="kimi-k3",  # per the live /models catalog
    api_key="YOUR_API_KEY",
    base_url="https://exaiv.com/api/ai/v1",
)

# Define tools
def search_docs(query: str) -> str:
    """Search the knowledge base"""
    return f"Search results for: {query}"

tools = {"search_docs": search_docs}

# Agent node
def agent_node(state):
    messages = state["messages"]
    response = llm.invoke(messages)
    return {"messages": [response]}

# Tool execution node
def tool_node(state):
    last_msg = state["messages"][-1]
    results = []
    for call in last_msg.tool_calls or []:
        result = tools[call["name"]](**call["args"])
        results.append(f"Tool {call['name']} returned: {result}")
    return {"messages": [{"role": "tool", "content": "\n".join(results)}]}

# Build the state graph
graph = StateGraph(MessagesState)
graph.add_node("agent", agent_node)
graph.add_node("tools", tool_node)
graph.set_entry_point("agent")

# Conditional routing: if tool calls exist, go to tools; otherwise end
def should_continue(state):
    last = state["messages"][-1]
    if hasattr(last, "tool_calls") and last.tool_calls:
        return "tools"
    return END

graph.add_conditional_edges("agent", should_continue)
graph.add_edge("tools", "agent")  # tool results return to the agent
app = graph.compile()

# Run
result = app.invoke({"messages": [HumanMessage(content="Search for the latest Qwen information")]})
print(result["messages"][-1].content)

Multi-model collaboration: different models for different steps

Agent steps have different model requirements — this is the best scenario for combining open Chinese models:

# Orchestration decisions use GLM (steady tool calling)
orchestrator = ChatOpenAI(
    model="glm-5.3-flash",
    api_key="YOUR_API_KEY",
    base_url="https://exaiv.com/api/ai/v1",
)

# Long-document analysis uses Kimi (long context)
reader = ChatOpenAI(
    model="kimi-k3",
    api_key="YOUR_API_KEY",
    base_url="https://exaiv.com/api/ai/v1",
)

# Multi-agent collaboration
def analysis_agent(state):
    """Read documents and analyze"""
    docs = state.get("documents", "")
    response = reader.invoke([HumanMessage(content=f"Analyze this document:\n{docs}")])
    return {"analysis": response.content}

def planning_agent(state):
    """Plan based on analysis"""
    analysis = state.get("analysis", "")
    response = orchestrator.invoke([HumanMessage(content=f"Plan based on this analysis:\n{analysis}")])
    return {"plan": response.content}

Three error-recovery essentials

1. Tool failure is not fatal. Return the error to the model so it knows the call failed and can try another approach. Do not let exceptions break the loop.

2. Timeout degradation. If a step hangs (e.g., an external API unresponsive), set a timeout and inform the model, rather than waiting forever.

3. Result size control. Large tool results (like a full database query) should be summarized or truncated before being returned to the model to avoid context explosion.

From single agent to multi-agent

A single agent handles most scenarios. When you need multiple specialized agents collaborating, the advanced patterns are:

Choosing between local and cloud agents

The same agent code can run on local models (via Ollama's OpenAI-compatible endpoint) or cloud models (via EXAIV). The choice depends on your requirements:

Local agent advantages: data never leaves your machine, zero per-call cost, works offline. The tradeoff: tool-calling stability may be lower on smaller local models, requiring more careful prompt design and testing.

Cloud agent advantages: better tool-calling reliability with flagship models, higher context capacity, faster responses. The tradeoff: per-call cost and data leaving your machine.

A practical hybrid: run the planning and orchestration steps on a cloud model (where tool-calling stability matters most), and route the heavy document analysis steps to a local model (where privacy and zero cost matter most). The routing logic is the same pick_model function described in the model selection article.

How to handle common agent patterns

Three patterns appear repeatedly in agent design:

1. Search-then-answer. The agent searches for relevant information, then answers. This is the simplest pattern — a RAG tool plus a generation step. Most chat-with-docs products are this pattern with a nice UI.

2. Plan-execute-verify. The agent creates a plan, executes each step, and verifies the result before moving on. More reliable for complex tasks but slower. Good for code changes, research reports, and multi-step data analysis.

3. Parallel dispatch. The agent splits a task into subtasks, runs them in parallel (each subagent handles one), then combines results. This is the most complex pattern but gives the best throughput for independent subtasks. Use it when subtasks truly have no dependencies — otherwise the coordination cost outweighs the speed gain.

Each pattern changes the state graph structure but the core loop (model decides → tool executes → feedback) remains the same.

How to handle common agent patterns

Three patterns appear repeatedly in agent design, and understanding them helps you choose the right structure before writing code:

1. Search-then-answer. The agent searches for relevant information, then answers. This is the simplest pattern — a RAG tool plus a generation step. Most chat-with-docs products are this pattern with a nice UI. The search tool typically wraps a vector store (Chroma, Qdrant) or keyword search over your documents.

2. Plan-execute-verify. The agent creates a plan, executes each step, and verifies the result before moving on. More reliable for complex tasks but slower. Good for code changes, research reports, and multi-step data analysis. The verification step is critical — it catches errors early rather than letting them propagate.

3. Parallel dispatch. The agent splits a task into subtasks, runs them in parallel, then combines results. This gives the best throughput for independent subtasks. Use it when subtasks truly have no dependencies — otherwise the coordination cost outweighs the speed gain.

Each pattern changes the state graph structure but the core loop remains the same: the model decides what to do next, the framework executes, and results feed back into the model.

Production readiness checklist

Moving from demo to production requires checking several items that are easy to skip during prototyping:

1. Tool permission scoping. Each tool should explicitly limit what resources it can access, what parameters it accepts, and what identity it runs under. Never give the model unrestricted shell access or database credentials — even if it seems convenient during development.

2. Input validation. Tool arguments are generated by the model and must be validated as if they were user input — especially URLs, file paths, SQL queries, and external API parameters. A hallucinated file path or SQL injection from the model is a real risk.

3. Maximum cost per task. Set a maximum number of rounds, tokens, and tool calls per task. This prevents anomalous loops from consuming your quota and makes the cost per task predictable.

4. Observability. Log which tools were selected each round, their execution time, result size, and the final answer. But be careful not to write sensitive content into unprotected logs — the model may process confidential data during tool calls.

5. Human-in-the-loop for irreversible actions. For actions like sending emails, modifying production data, or making payments, add a human confirmation node in the state graph. This is the simplest way to prevent costly mistakes.

6. Replay testing. Prepare a fixed set of tasks and replay them after every model change or prompt update. Observe whether tool selection and final results have degraded — this catches regressions that a single manual test would miss.

Agent quality is not just about the final answer. Wrong tool selection, excessive retries, and context expansion are all metrics that need separate monitoring and alerting.

FAQ

Related reading

• Qwen Streaming and Tool Calling — tool-calling basics
• RAG Knowledge Base Guide — agent plus RAG combination
• Qwen Local Coding Workflow — local agent setups
• Qwen Cost Optimization — controlling agent token consumption
• Pricing — live pricing