Editor and IDE Integration on EXAIV: Building a Custom AI Assistant on the Gateway

Editor assistants are a common target for a gateway, because the integration surface is narrow and the usage pattern is repetitive: a piece of context goes in, a piece of text comes out. A custom assistant also avoids being at the mercy of a third-party extension's pricing, its data handling, and its fixed model choices.

Building one on EXAIV takes three parts: a client that talks to the OpenAI-compatible endpoint, a way to decide what context to send, and an interface for reviewing what comes back. The second part is where most of the engineering actually lives — the model is rarely the hard part.

Architecture: why the key belongs in a backend

The obvious design is an extension that calls the endpoint directly. It is simpler to build, and it puts a credential into a binary that runs on every developer machine, is trivially inspectable, and will eventually be shared, logged, or extracted.

The design that ages better:

Editor extension  ──HTTPS──>  Your backend  ──HTTPS──>  EXAIV endpoint
   (no credentials)            (holds the key)              (model access)

The backend earns its keep by doing the things the extension should not:

• holding and rotating the key;
• deciding which context to send and how much of it;
• enforcing per-user limits and recording usage;
• caching repeated context, and logging what was sent.

The extension becomes a thin client: it sends the user's selection and the active file's language, and renders the streamed response.

Choosing the context

This is the part that determines whether the assistant is useful or annoying. The model cannot see your screen.

Three rules that keep the prompt useful:

1. Send the least that works. A selection with the surrounding function is often enough; a whole file is rarely better and always costs more.
2. Label the context. Mark each block with its source so the model can refer to it and so you can debug what was sent.
3. Never send secrets. .env files, key files, and credential stores should be filtered before the request leaves your backend. This filtering belongs on the server, where it cannot be bypassed by a user or a compromised client.

Building the request

A request that carries a selection and the surrounding function:

messages = [
    {"role": "system", "content": EDITOR_SYSTEM_PROMPT},
    {"role": "user", "content": build_prompt(
        language=context.language,
        path=context.path,
        selection=context.selection,
        surrounding=context.surrounding,
    )},
]

response = client.chat.completions.create(
    model="gpt-6-sol",
    messages=messages,
    max_tokens=1024,
    stream=True,
    temperature=0,
)

The system prompt should set the boundaries explicitly: answer about the provided context, say so when the context is insufficient, keep responses short, and do not invent API names that are not in the context. A short, precise system prompt does more for output quality than a long one.

Streaming into the editor

Streaming matters more here than in a typical application. A user asking a question in an editor is waiting in a context where a spinner feels broken.

for chunk in stream:
    delta = chunk.choices[0].delta
    if delta.content:
        yield delta.content

In the extension, append each chunk to the output panel as it arrives. Two implementation details:

• Render incrementally, commit at the end. Appending to a DOM buffer per chunk is fine for a typical answer; for very long answers, batch the updates so the UI does not thrash.
• Detect an incomplete stream. A stream that ends without a finish reason is a partial answer. Mark it as such rather than presenting it as complete. The streaming guide covers the detection.

Propose, then apply

An editor assistant that writes files without showing you is a liability. The safe pattern:

1. The model returns a diff or a new file body, never a direct write.
2. The extension renders it as a reviewable change.
3. The user accepts or rejects it.

This mirrors how a good code review works, and it keeps the recovery path trivial: reject the change and nothing happened.

If your assistant can only edit through a tool call, apply the same rule in the tool implementation — return the proposed change as data, and let the extension decide whether to write it.

Latency and cost in an editor

Two editor-specific realities:

Latency is perceived. The time to first token matters more than the total time. A short answer that starts instantly feels better than a marginally better answer that arrives after a long pause. Consider a faster tier for the first response and a stronger one for follow-up turns if your budget allows.

Cost accumulates invisibly. A user who holds a key combination and asks about the whole file will spend a lot without noticing. Give the assistant an explicit budget per session, and make the current spend visible in the UI.

Two further levers:

• Cache the stable context. The system prompt and the project structure are stable across calls; the prompt caching guide explains how to structure them so they can be reused.
• Choose the tier per task. "Explain this function" and "refactor this module" do not need the same model. See the GPT model selection guide.

A simple command surface

Most editor assistants work better with a few explicit commands than with a free-form chat box:

Each command maps to one prompt template, one context selection rule, and one output mode. That structure is far easier to evaluate and to keep consistent than a single prompt that handles everything.

Reference implementation sketch

from fastapi import FastAPI

app = FastAPI()

@app.post("/assistant")
async def assistant(req: AssistantRequest):
    messages = build_messages(req.context)          # includes server-side filtering
    stream = await client.chat.completions.create(
        model=req.model or "gpt-6-sol",
        messages=messages,
        max_tokens=req.max_tokens or 1024,
        stream=True,
        temperature=0,
    )
    async def gen():
        async for chunk in stream:
            delta = chunk.choices[0].delta
            if delta.content:
                yield f"data: {json.dumps({'text': delta.content})}\n\n"
    return StreamingResponse(gen(), media_type="text/event-stream")

The extension receives a small event stream and renders it. The key never leaves the server, the context filtering happens in one place, and usage is recorded per user.

Privacy, telemetry, and consent in editor assistants

An editor assistant reads whatever is open on a developer's screen, including source code that may be under a confidentiality agreement. Three decisions should be made explicitly rather than inherited by default.

What is sent. Filter on the server, not in the extension. Exclude files by pattern, exclude directories, and exclude anything that looks like a credential. The filter belongs in the one place that every client goes through, because a client-side filter can be bypassed and a new client will forget it.

What is logged. Log the metadata needed to operate the service — request ID, model, token counts, latency — and avoid logging full prompt and completion content unless there is a specific reason. Content logs are a second copy of your users' code, stored somewhere with a different access policy.

What is shown to users. An assistant that quietly reads the whole workspace is different from one that says which files it used. Surfacing the context in the interface makes the behavior legible and gives users a chance to notice something wrong.

These decisions also belong in whatever terms or internal policy you publish, because a developer assistant running against a company codebase is a data path that someone will eventually ask about.

IDE integration checklist

• The key lives in a backend, never in the extension or in client-side storage.
• Context selection rules are defined per command, with a size budget.
• Sensitive files and patterns are filtered on the server before the request.
• Responses are streamed and rendered incrementally.
• An incomplete stream is detected and marked as partial.
• Changes are proposed as diffs and applied only after user confirmation.
• The model and parameters used are recorded with each suggestion.
• Per-user usage is tracked and a budget is visible in the interface.
• Each command has its own prompt template and can be evaluated independently.

Further reading

• Coding Agent Integration Guide — the same gateway powering a full agent loop.
• Streaming Responses Guide — the streaming path an editor depends on.
• API Key Management and Rotation — keeping the assistant's key safe.

FAQ

Related reading

• Cursor with EXAIV
• Continue with EXAIV
• Qwen Local Coding Workflow