how-tos

Gemini 4 Carbon API for Coding Agents: A Bounded Agent Loop with Streaming and Cost Guards

Poyo.ai Team
10 min read
Share:

Gemini 4 Carbon API powering a coding agent loop

The Gemini 4 Carbon API is Google's pick for its hardest coding work. Google built it for multi-step debugging and long-horizon agentic coding, and says it spends more tokens than Argon to explore, reason, and run verification loops. This guide wires gemini-4-carbon (model page) into a coding agent through PoYo's OpenAI-compatible endpoint. The agent streams its output, can read a large slice of the repository, stops at a hard turn and dollar budget, and knows when to hand work to Argon.

If you haven't sent a first request yet, start with the Gemini 4 Carbon API guide.

What the API gives a coding agent

ItemGemini 4 Carbon on PoYo
Model IDgemini-4-carbon
Agent endpoint used herePOST https://api.poyo.ai/v1/chat/completions (OpenAI SDK, base_url="https://api.poyo.ai/v1")
Native alternativePOST /v1beta/models/gemini-4-carbon:generateContent and :streamGenerateContent
Context window1M tokens: text, image, audio, video, PDF
Max output per response64K tokens
Streaming"stream": true; the last chunk carries finish_reason and usage
Price$1.60 input / $8.00 output / $0.08 cache read per 1M tokens (320 / 1,600 / 16 credits)
Rate limitsPoYo's docs state LLM endpoints are not currently rate limited
Errors400 invalid fields, 401 bad key, 402 insufficient credits, 404 unknown model; retry 429 / 500 / 502 / 503 with backoff

The Chat Completions docs document these request fields: model, messages (string content with system, user, or assistant roles), max_tokens, stream, temperature, top_p, stop, n, and the penalty fields. The agent below uses only those.

Tool calling: what is documented today

PoYo documents tools for Anthropic Messages and for Responses. It does not document tools or function declarations for Gemini models, in either Chat Completions or Gemini Native. Responses isn't available for gemini-4-carbon.

So this agent doesn't send a tools array. It uses a JSON action protocol in plain message content instead. The system prompt lists the allowed actions, the model replies with one JSON object per turn, and your code runs that action and sends the result back as the next user message. This works with any chat model. When PoYo documents native function calling for Gemini, you can swap the parser for real tool calls and keep the rest of the loop.

There's a safety benefit too: the model can only request four actions you wrote yourself. It can't run an arbitrary shell command.

Long-context repo input: 1M in, 64K out

A 1M-token window lets you send a large part of the repository on the first turn instead of fetching files one at a time. Google's launch table gives a reason to use it: Carbon scores 96.2% on GraphWalks at 256K–1M tokens, against 84.2% for Argon. Three limits shape the loop:

  • Every turn resends the whole history. The snapshot is billed again on each call. A 300K-token snapshot over 12 turns is at least 3.6M input tokens, which is $5.76 at $1.60 per 1M before any output.
  • Output is capped at 64K per response. Ask for one file per write_file action, not a whole-repo rewrite in one reply.
  • Measure, don't guess, token counts. PoYo documents no tokenizer endpoint. Read usage.prompt_tokens from the first call, then tune MAX_SNAPSHOT_CHARS to the token budget you want.

Caching. PoYo's price table has a cache-read rate for Carbon ($0.08 per 1M, versus $1.60 for fresh input), and Google prices cached input at 95% off. PoYo's docs don't document a caching request parameter or a cached-token usage field for Gemini models. This loop sends no caching parameter and budgets every prompt token at the full input rate. Keeping the system prompt and snapshot as an unchanged prefix costs nothing, and your bill will show whether any reads were charged at the cache rate.

The bounded agent loop in Python

import json
import os
import subprocess
import time
from pathlib import Path

from openai import OpenAI, APIStatusError

client = OpenAI(api_key=os.environ["POYO_API_KEY"], base_url="https://api.poyo.ai/v1")

CARBON, ARGON = "gemini-4-carbon", "gemini-4-argon"
PRICE_IN, PRICE_OUT = 1.60, 8.00      # PoYo USD per 1M tokens, same for both (2026-11-16)

MAX_TURNS = 12
BUDGET_USD = 2.00
MAX_TOKENS_PER_TURN = 16_000          # Carbon allows up to 64,000
MAX_SNAPSHOT_CHARS = 400_000          # tune against usage.prompt_tokens
MAX_RESULT_CHARS = 20_000
REPO = Path("./my-repo").resolve()

SYSTEM = """You are a coding agent working in a Python repository.
Reply with exactly one JSON object and nothing else. Allowed actions:
{"action": "read_file", "path": "<relative path>"}
{"action": "write_file", "path": "<relative path>", "content": "<full file content>"}
{"action": "run_tests"}
{"action": "done", "summary": "<what changed and why>"}"""


def snapshot() -> str:
    parts, total = [], 0
    for p in sorted(REPO.rglob("*.py")):
        block = f"### {p.relative_to(REPO)}\n{p.read_text(encoding='utf-8', errors='replace')}\n"
        if total + len(block) > MAX_SNAPSHOT_CHARS:
            break
        parts.append(block)
        total += len(block)
    return "".join(parts)


def safe_path(rel: str) -> Path:
    p = (REPO / rel).resolve()
    if not p.is_relative_to(REPO):
        raise ValueError("path outside repository")
    return p


def execute(action: dict) -> str:
    kind = action.get("action")
    if kind == "read_file":
        return safe_path(action["path"]).read_text(encoding="utf-8", errors="replace")
    if kind == "write_file":
        safe_path(action["path"]).write_text(action["content"], encoding="utf-8")
        return "written"
    if kind == "run_tests":
        r = subprocess.run(["pytest", "-q"], cwd=REPO, capture_output=True, text=True, timeout=600)
        return f"exit={r.returncode}\n{r.stdout}\n{r.stderr}"
    raise ValueError(f"unknown action {kind!r}")


def call(model: str, messages: list):
    """One streamed turn. Returns (text, usage, finish_reason)."""
    for attempt in range(4):
        try:
            stream = client.chat.completions.create(
                model=model, messages=messages, max_tokens=MAX_TOKENS_PER_TURN, stream=True
            )
            text, usage, finish = [], None, None
            for chunk in stream:
                if chunk.choices:
                    choice = chunk.choices[0]
                    if choice.delta and choice.delta.content:
                        text.append(choice.delta.content)
                        print(choice.delta.content, end="", flush=True)
                    finish = choice.finish_reason or finish
                if getattr(chunk, "usage", None):
                    usage = chunk.usage
            return "".join(text), usage, finish
        except APIStatusError as e:
            if e.status_code in (429, 500, 502, 503) and attempt < 3:
                time.sleep(2 ** attempt)
                continue
            raise  # 400, 401, 402, 404: fix the request or the account, don't retry


def parse(text: str) -> dict:
    t = text.strip()
    if t.startswith("```"):
        t = t.strip("`").removeprefix("json").strip()
    return json.loads(t)


def run_agent(task: str, model: str = CARBON) -> dict:
    messages = [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": f"Repository snapshot:\n{snapshot()}\n\nTask: {task}"},
    ]
    spent, last_prompt = 0.0, 0
    for turn in range(1, MAX_TURNS + 1):
        # Worst case for the next turn: at least last turn's input plus a full output
        worst_next = last_prompt / 1e6 * PRICE_IN + MAX_TOKENS_PER_TURN / 1e6 * PRICE_OUT
        if spent + worst_next > BUDGET_USD:
            return {"status": "budget_stop", "turns": turn - 1, "spent_usd": round(spent, 4)}

        text, usage, finish = call(model, messages)
        if usage is None:
            raise RuntimeError("no usage in stream; stopping so the budget stays enforceable")
        spent += usage.prompt_tokens / 1e6 * PRICE_IN + usage.completion_tokens / 1e6 * PRICE_OUT
        last_prompt = usage.prompt_tokens
        messages.append({"role": "assistant", "content": text})

        if finish == "length":
            messages.append({"role": "user", "content": "Reply was cut off. Send one shorter JSON action."})
            continue
        try:
            action = parse(text)
        except json.JSONDecodeError:
            messages.append({"role": "user", "content": "Reply with exactly one JSON object."})
            continue
        if action.get("action") == "done":
            return {"status": "done", "summary": action.get("summary"), "turns": turn, "spent_usd": round(spent, 4)}
        try:
            result = execute(action)
        except Exception as e:  # report the failure to the model instead of crashing
            result = f"error: {e}"
        messages.append({"role": "user", "content": f"Result of {action.get('action')}:\n{result[:MAX_RESULT_CHARS]}"})

    return {"status": "max_turns", "turns": MAX_TURNS, "spent_usd": round(spent, 4)}


if __name__ == "__main__":
    print(run_agent("tests/test_parser.py::test_unicode fails. Find the root cause and fix it."))

Path.is_relative_to needs Python 3.9 or later. Run the agent against a copy or a branch of the repository, since write_file overwrites files.

How the cost guards work

GuardWhat it stopsWhere the number comes from
MAX_TURNSLoops that never call doneYour choice; 12 is a starting point
BUDGET_USD + worst-case checkSpending past the cap on the next callusage.prompt_tokens and usage.completion_tokens from the final stream chunk, at PoYo's $1.60 / $8.00
MAX_TOKENS_PER_TURNOne runaway replymax_tokens, at most 64,000 for Carbon
MAX_RESULT_CHARSHuge test logs inflating every later turnTruncation before the result enters history
usage is None → stopSilent spending you can't measurePoYo documents usage on the last stream chunk
402 not retriedRetry storms on an empty balancePoYo error codes

Google prices Carbon's output including thinking tokens, so a turn can bill more output than the visible JSON. That's why the budget uses completion_tokens and never counts the characters you received. The estimate counts all input as uncached, so it's an upper bound on what PoYo will bill. Check it against your dashboard for the first few runs.

The worst-case check is conservative. With a 300K-token history, one more turn costs at least $0.48 of input plus up to $0.128 of output at a 16K max_tokens. On a $2.00 budget, the loop stops before the call that could cross the cap.

When to fall back to Gemini 4 Argon

Argon takes the same request and costs the same per input and output token. Google calls it "the ideal choice for the majority of software engineering and knowledge work," with faster responses and lower token spend per turn. Route to it with run_agent(task, model=ARGON) when:

  • The task is routine. Small fixes, single-file changes, and test writing don't need Carbon's extra verification tokens.
  • One response must exceed 64K tokens. Carbon caps output at 64K. Our Argon guide covers Argon's much larger output limit.
  • You want a cheap first attempt. Run Argon first and escalate to Carbon only when the tests still fail. The router in Gemini 4 Carbon API vs Gemini 4 Argon API does this.

Keep Carbon for repository-wide root-causing and terminal-heavy loops. In Google's table, Carbon scores 65.2% on Terminal-Bench 4.0 (AA) against Argon's 57.1%, and 9.6% on SWE-sweep against 4.3%. Google's availability post also suggests a split: Carbon for architecture, coordination, and auditing roles, paired with faster, lighter models for parallel work. In this loop, that means one Carbon planner run that assigns files to Argon workers.

Budgets for both models are in the Gemini 4 Carbon API pricing post. If you're weighing other vendors for the agent, see Gemini 4 Carbon API vs Claude Opus 5.5 API vs GPT-6 Astra API.

Frequently asked questions

Does the Gemini 4 Carbon API support function calling on PoYo?

PoYo's current docs don't document tools or function declarations for Gemini models in Chat Completions or Gemini Native. Tools are documented for Anthropic Messages and Responses. This guide uses a JSON action protocol in message content, which needs only documented fields.

Can I use the OpenAI Python SDK with the Gemini 4 Carbon API?

Yes. Create OpenAI(api_key=..., base_url="https://api.poyo.ai/v1") and pass model="gemini-4-carbon" to chat.completions.create. Streaming works with stream=True, and the last chunk carries usage.

What is the Gemini 4 Carbon API max output and context window?

1M tokens of context and up to 64K output tokens per response. Set max_tokens to 64,000 or less.

How do I limit Gemini 4 Carbon API costs in an agent loop?

Cap turns, cap max_tokens per turn, truncate tool results, and stop when the spend so far plus a worst-case next turn would pass your budget. Compute spend from usage.prompt_tokens × $1.60 and usage.completion_tokens × $8.00 per 1M tokens on PoYo.

Does the Gemini 4 Carbon API support context caching?

PoYo lists a cache-read price for Carbon of $0.08 per 1M tokens (16 credits). Its docs don't currently document a caching request parameter or a cached-token usage field for Gemini models, so budget prompts at the full input rate and check your bill for cache-rate charges.

When should a coding agent use Gemini 4 Argon instead of Carbon?

Use Argon for routine coding, for any single response that must exceed 64K tokens, and as a cheaper first attempt before you escalate to Carbon. Both cost $1.60 / $8.00 per 1M tokens on PoYo, but Google says Argon uses fewer tokens per turn.

Share: