Chat Completions Parameters Reference for EXAIV: What Each Field Does and When to Change It

Most integrations send a request with model, messages, and max_tokens, and never look at the rest of the request body. That is usually fine — but the parameters you leave alone are also the ones that decide whether your output is consistent, whether it is truncated, whether it costs more than you expected, and whether it can be parsed by the system downstream.

This is a reference to the parameters that matter in practice, what each one does, and when changing it is worth the risk. It is not a restatement of every field in the API; it is the working subset, with the reasoning attached.

The request body

A complete, ordinary request:

{
  "model": "gpt-6-sol",
  "messages": [
    {"role": "system", "content": "You are a precise technical editor."},
    {"role": "user", "content": "Rewrite this paragraph."}
  ],
  "max_tokens": 1024,
  "temperature": 0,
  "stream": false
}

messages and roles

The conversation is an array of messages, each with a role and content.

Two things are commonly misunderstood:

• The system role is a prompt, not a sandbox. It is a strong instruction, but it is not a guarantee. If a workflow depends on the system message being followed, verify the output rather than assuming it.
• Replayed assistant turns are your responsibility. When you send prior assistant messages back as context, you are asserting that the model produced them. If you filtered or edited the history, send what the user actually saw.

Content can be a plain string or an array of content parts, which is how images and other media are passed. Keep the plain-string form unless you need the richer structure.

max_tokens

The output ceiling. It is the single most important parameter to set deliberately, because the failure mode is quiet: the response stops mid-sentence, finish_reason is length, and the content still looks like a valid answer if you do not check.

The cost relationship matters: output tokens are typically priced above input tokens, so an unnecessarily large max_tokens is a budget reservation, not a target. The model stops when it finishes, so a generous ceiling costs nothing unless the model actually uses it — but it does allow a runaway response to get expensive before you notice.

temperature and top_p

Both control randomness, and they are not independent knobs.

Practical guidance:

• temperature: 0 for classification, extraction, translation of fixed strings, code generation, and anything you evaluate. It is the closest thing to deterministic the API offers.
• Leave top_p at its default unless you have a specific reason. Changing both temperature and top_p at once is the classic mistake: the interaction is not intuitive and the result is hard to reason about.
• Moderate temperature for creative work where variety is the point.
• Low temperature for factual extraction, where a "creative" answer is simply wrong.

Change one of them, measure, and keep the result in version control.

stop sequences

stop ends generation when the model produces one of the given strings.

{"stop": ["\n\n", "END"]}

It is useful for structured formats — the model stops before it writes the next section — but it is dangerous with ordinary prose, because stop matches text anywhere, including inside a legitimate answer. If a response ends abruptly at a strange place, check your stop list before you blame the model.

Penalties

presence_penalty and frequency_penalty nudge the model away from tokens it has already used.

Both default to 0 and both are negative in normal use — you want repetition reduced, not encouraged. They are occasionally useful for reducing loops in a retry, but they are not a substitute for fixing a prompt that asks the same question twice. Values outside a small range produce degraded text, and the exact useful range is model-dependent.

seed

seed requests reproducibility for the same prompt and parameters. Treat it as a request, not a guarantee — a request routed to a different backend or a model update can change the output anyway. It is useful for regression tests and for reproducing a reported failure, not as a guarantee for production determinism.

stream

stream: true returns incremental events instead of one payload. It changes latency characteristics, not the content. The mechanics, including how to detect truncation and what buffering does to the experience, are covered in the streaming guide.

n and best_of

n asks for several candidate completions; best_of asks the server to generate several and return the best. Both multiply cost and latency. Use n when you want several options to present or to select between; leave best_of alone unless you know the difference is worth several times the work.

response_format

response_format controls the shape of the output, which is what makes a response parseable by code instead of by a hopeful regular expression.

For anything that feeds a parser, set this rather than instructing the model in prose to "reply with JSON only". A schema also documents the contract in the request, which makes the expectation enforceable. The full pattern, including validation and retry, is in the structured output guide.

tools and tool_choice

tools declares callable functions; tool_choice decides whether the model must call one. They are the mechanism behind agent loops and structured extraction. See the function calling guide.

user

user is an end-user identifier that the platform can use for abuse monitoring and rate limiting. It is not a security boundary and it does not authenticate anything. Set it when you have a stable identifier for the end user; it helps the platform apply limits fairly.

A minimal request template

Most applications do not need more than a handful of parameters. Starting from a small request and adding fields with a reason in mind is easier to maintain than copying a large template and deleting the parts you did not understand.

{
  "model": "gpt-6-sol",
  "messages": [{"role": "user", "content": "..."}],
  "max_tokens": 1024,
  "temperature": 0
}

That is the entire request for a large share of workloads. Add a parameter only when one of these is true:

• the output feeds a parser, and you need response_format;
• the model must call something, and you need tools;
• the answer must arrive incrementally, and you need stream;
• you have measured that the default sampling behaviour is wrong for this workload.

Every additional parameter is a decision someone has to understand, explain, and eventually debug. A parameter list that grew by accident is the one nobody can reason about when the output suddenly changes.

Parameters worth leaving alone

A short list that saves time:

A note on compatibility

EXAIV serves the OpenAI Chat Completions protocol, so the fields above behave as documented. Not every model in the catalog supports every field: reasoning-oriented models in particular may handle temperature differently or ignore sampling knobs. When a parameter appears to have no effect on one model but works on another, that is a model capability difference, not a gateway bug.

Parameter checklist

• model is an exact ID from the live catalog.
• max_tokens is set deliberately and sized to the expected output.
• finish_reason is checked, and length is treated as truncation.
• temperature: 0 is used for anything measured, classified, or parsed.
• top_p is left alone unless a measurement justified changing it.
• stop sequences do not match text that can legitimately appear in the answer.
• response_format is set when the output feeds a parser.
• Sampling parameters are changed one at a time and the change is recorded.
• Fields the workload does not need are left out of the request.

Further reading

• JSON Structured Output Guide — making the response reliably parseable.
• Function Calling Guide — tools and tool choice.
• GPT Model Selection Guide — the one parameter that changes everything else: which model you call.

FAQ

Related reading

• Streaming Responses Guide
• JSON Structured Output Guide
• Function Calling Guide

Terminology and pre-flight checklist

Total parameters, active parameters and the KV cache are three different quantities. Total parameters are all the weights that must be loaded into memory; active parameters are the subset that computes a single token. KV cache is temporary memory that grows linearly with context length, so a 1M window is a ceiling rather than a free default. Quantization lowers the precision of stored weights to shrink files; it does not reduce the parameter count. A vision_config in a config.json describes architecture, not a wire format you may safely invent.

Before shipping anything built on these notes, run six checks. Re-read the numbers from the live source on the day you publish, because download counters move weekly. Open the live /models catalog and confirm the model ID you plan to call is still there. Open /pricing rather than copying a price into your application. Never hard-code a request body you have not validated against the official reference. Record the model ID, input and output token counts, and latency for a representative set of prompts, then keep that set unchanged so a later model swap produces a comparable reading. Read the licence on the model card before commercial use, since vendor-specific terms are not a permissive default. Add exponential backoff for retryable requests, set a timeout, and log the error class so that a 401, a 404 and a 429 never get the same treatment.