Coding Agents on EXAIV: Pointing CLI Agents at an OpenAI-Compatible Endpoint

A coding agent is a loop: it reads your repository, calls a model, receives instructions to run a command or edit a file, runs it, and calls the model again with the result. That loop is far more expensive and far more invasive than a chat request, because each turn carries tool output into the next prompt and because the agent can change your working tree.

Running one against a gateway instead of a first-party endpoint is a normal and useful setup — it is how you get a specific model, a controlled cost, or a route you trust. The integration is small: point the agent at an OpenAI-compatible base URL, give it a key, and choose a model. The parts that need care are what the agent does with your repository while it is running.

The integration surface

Every OpenAI-compatible agent ultimately needs three things:

Most agents read the first two from environment variables that already exist in the OpenAI ecosystem, so the setup is often a matter of setting them in the shell that launches the agent:

export OPENAI_BASE_URL="https://exaiv.com/api/ai/v1"
export OPENAI_API_KEY="YOUR_EXAIV_API_KEY"

If the agent accepts a model override, use a catalog ID. If it does not, set the model in the agent's own configuration. Either way, the ID has to be exact — the same rule that applies to a direct SDK call applies here, and a wrong ID surfaces as a model-not-found error on the agent's first turn.

The part that is not a configuration detail

An agent that can read files can also rewrite them, run shell commands, and install packages. Before you point one at a repository you care about, decide three things.

Where it runs. A disposable branch or a separate worktree means a destructive edit is recoverable:

git switch -c agent/scratch
# run the agent here
git diff          # inspect what changed
git switch -     # back to the original branch, changes discarded

What it may touch. If the agent supports permission rules, start with the narrowest useful set — read-only file access, and no writes outside the project directory. Widen the permissions only for tasks where you need them, and reset them afterwards.

How it is observed. A coding session can run for many turns. Watch the output while it works. An agent that starts rewriting files it was not asked to touch is showing you something you want to know about early.

Verifying the connection before the real task

Run the agent in a throwaway directory with a trivial task. The goal is not the answer; the goal is to confirm that the agent can reach the endpoint, that the model ID resolves, and that the tool loop functions.

mkdir -p /tmp/agent-check && cd /tmp/agent-check
printf 'def add(a, b):\n    return a + b\n' > calc.py

Ask the agent to read calc.py and explain what it does. Then check:

• The agent reached the endpoint and produced a response.
• The model ID resolved — no model-not-found error.
• The agent read the file it was asked to read.
• Nothing was written or executed beyond what you allowed.
• Usage appears in the console and the credit cost matches the number of turns.

If the agent fails here, the problem is the connection or the model ID, and nothing about the repository is relevant yet. Fix that first.

Why agent sessions cost more than they look

A single agent turn is one API request, but a session is many. The reasons a session is expensive are worth understanding before you enable one:

• Context accumulates. Each turn appends tool output — file contents, command results, diffs — to the conversation. Later turns send all of it again. The per-request token count grows through the session.
• The agent retries. A failing command produces an error, the agent tries a fix, and the error text stays in the context. Errors are not free.
• Long tool outputs are expensive. Printing a large file or a verbose log can cost more than the instruction that requested it.

Practical mitigations, in rough order of impact:

1. Give the agent a narrow task. A precise instruction ends the session sooner than a broad one.
2. Keep the working directory small. Irrelevant files in scope are context the agent may read.
3. Cap the session. If your agent supports a turn or token limit, set it — an unbounded session on a complicated repository can run for a very long time.
4. Choose a tier that fits the work. Routine edits and file navigation rarely need the strongest model. See the GPT model selection guide.

Streaming matters more for agents

Agents render their reasoning and tool calls incrementally, so a non-streaming agent looks frozen for the entire duration of a turn and then dumps everything at once. If the agent supports streaming, use it.

If it does not, the practical workaround is to run it in a terminal that shows output as it arrives, and to give the agent tasks whose turns are short. The streaming responses guide covers the wire format and how to detect a stream that ends early — which is the failure mode that matters most in a long agent session.

Tool calling and function schemas

Agents call tools, and tool definitions travel with every request. Two practical consequences:

• Tool schemas are part of your per-request cost. Large schemas, long descriptions, and many tools increase the input tokens of every turn.
• Not every model routes tool calls identically. A model may emit a tool call with slightly different argument types or omit optional fields. If an agent behaves inconsistently across models, compare the raw tool-call output before blaming the prompt.

A function-calling walkthrough, including how to execute tool calls and feed results back, is in the function calling guide.

Cost control for agent sessions

An agent session is the most expensive way to use a model, and the cost is not where people expect it. The prompt is rarely the main expense — the accumulated tool output is. Every command result, file read, and diff stays in the conversation and is re-sent on every later turn.

Three levers, in order of impact:

1. Bound the tools, not just the task. A command that prints a thousand lines is the single most expensive event in most sessions. Narrow the tool output before it reaches the model: tail long output, cap the file size, and summarize instead of returning raw pages.
2. Cap the rounds. A session that has made eight tool calls without progress is usually stuck. Stop it and report where it got to, rather than letting it spend another twenty calls.
3. Choose the tier per phase. Exploration and file navigation rarely need the strongest model. Use the stronger tier for the parts that genuinely need reasoning and the cheaper one for the mechanical steps.

Track credits per session, not per request. A session-level number is what tells you whether the feature is viable at all; a per-request number hides the compounding that makes agent products expensive.

Managing risk in a long session

The last row is the important one. An agent with network access and shell access can affect systems outside the repository. If your workflow includes deployment, migrations, or anything touching shared infrastructure, make that step manual.

A safe first session

A reasonable progression:

1. Empty directory, trivial task. Confirm connectivity and tool use.
2. A real repository, read-only. Ask for an explanation or a review. See how it reads code and how it reports.
3. A real repository, one small write. A single focused change, reviewed as a diff.
4. A branch, a multi-step task. Real work, recoverable.
5. Wider permissions. Only for the specific tasks that need them, and only after the behavior is familiar.

Each stage should be boring before you move to the next.

Integration checklist

• The agent is confirmed to support a custom OpenAI-compatible base URL and API key.
• The base URL is https://exaiv.com/api/ai/v1 and the model ID is copied from the live catalog.
• The connection was verified in an empty directory before touching a real repository.
• The session runs on a branch or worktree.
• Permissions start at the narrowest useful scope.
• Turn or token limits are configured.
• Streaming is enabled where supported.
• Usage per session is tracked against a budget.
• Anything reaching shared infrastructure requires a human step.

Further reading

• GPT Model Selection Guide — choosing a tier that fits agent work.
• Function Calling Guide — the tool-calling loop an agent runs on.
• API Key Management and Rotation — keeping the key the agent uses out of the repository it can read.

FAQ

Related reading

• IDE Integration Guide
• Qwen Local Coding Workflow
• Continue with EXAIV