How to implement Claude’s computer-use agent loop

Claude’s computer use tool is not a remote desktop session. It is a client toolset: you declare one entry in tools, Claude returns tool_use blocks for screenshot, click, type, and related members, and your application runs every call in an environment you control. Miss the loop—or forget a tool_result—and the API rejects the next turn or keeps sampling until you cap it.

This walkthrough follows Anthropic’s computer-use docs for the computer_toolset_20260801 contract on the Messages API. The toolset is not currently available in Claude Managed Agents. For tasks that stay inside webpages, the docs recommend the browser use tool instead of a full desktop.

Declare the desktop toolset

Add {"type": "computer_toolset_20260801"} to the tools array. The request needs no beta header. That single entry gives Claude 17 member tools such as screenshot, left_click, type, and zoom. Claude typically uses computer use alongside the text editor and bash tools; the quick start declares all three:

client = anthropic.Anthropic()

response = client.messages.create(
    model="claude-opus-5",
    max_tokens=1024,
    tools=[
        {"type": "computer_toolset_20260801"},
        {"type": "text_editor_20250728", "name": "str_replace_based_edit_tool"},
        {"type": "bash_20250124", "name": "bash"},
    ],
    messages=[{"role": "user", "content": "Save a picture of a cat to my desktop."}],
)
print(response)

When Claude acts on the desktop, stop_reason is tool_use. Each member block names the action and carries "toolset_name": "computer". A turn can include several of those blocks—a batch action.

Run the agent loop

The docs break the cycle into four steps:

Send the toolset and a user prompt that needs the desktop.
Claude replies with one or more member tool_use blocks.
Execute each call in order in your container or VM, then continue with a user message that has one tool_result per tool_use, matched by tool_use_id and echoing "toolset_name": "computer".
Claude either requests more actions (stop_reason of tool_use again) or returns text.

Repeating steps 3 and 4 without further user input is the agent loop. Anthropic’s sampling helper is the shape you want: call the API, append the assistant message, run tools, append results, and stop when there are no tool calls or you hit a max.

def sampling_loop(model: str, messages: list[MessageParam], max_iterations: int = 10):
    """
    Run the computer-use agent loop until Claude stops requesting tools
    or the iteration limit is reached.
    """
    for _ in range(max_iterations):
        response = client.messages.create(
            model=model,
            max_tokens=4096,
            messages=messages,
            tools=TOOLS,
        )

        # Add Claude's response to the conversation history
        messages.append({"role": "assistant", "content": response.content})

        # Run the actions Claude requested, in order, and collect the results
        tool_results = process_tool_calls(response)
        if not tool_results:
            return messages  # No more tool use; task complete

        # Send every result back to Claude in a single user message
        messages.append({"role": "user", "content": tool_results})

    return messages

max_iterations defaults to 10 in the example. That cap is the cost brake: without it, a stuck loop can keep calling the API.

If you also declare bash, text editor, or custom tools, dispatch those tool_use blocks in the same pass. The computer-only helper answers only members with toolset_name of computer. The loop treats a turn with no answered calls as finished.

Execute batches in order

A batch uses the same response shape as parallel tool use, with one difference: you run the blocks in order, not concurrently. Later actions usually depend on earlier ones—type enters text into whatever the preceding left_click focused.

Return one tool_result for every block, all in the next user message. Screenshot and zoom results need an image. Other members can return short text such as OK; cursor_position returns the coordinates as text.

Every result for a member tool must carry "toolset_name": "computer". A result that omits it, or that names a different toolset than its tool_use block, is rejected. Leave any block unanswered and the next request fails with invalid_request_error—so an agent loop that reads only the first block fails on its next call.

If one action fails, stop. Do not run the rest of the batch. Still answer every block: normal results for successes, is_error: true plus a description for the failure, and for every later member a tool_result in this shape from the docs:

{
  "type": "tool_result",
  "tool_use_id": "toolu_01Xf5W1sD8Q9aBcJ7kLmN2pQ",
  "toolset_name": "computer",
  "is_error": true,
  "content": "Not executed: an earlier computer action in this turn failed."
}

Claude then sees what succeeded, what failed, and what was skipped, and replans. If a human must confirm consequential actions, make that check before each block runs. A batch can complete a multistep action in one turn.

Claude typically finishes a batch with screenshot. When it does not, you can attach a screenshot as an extra image on the last result so the model always sees the current screen.

Dispatch on toolset name and member

There is no action field. Claude names the member in name; input holds only that member’s parameters. Dispatch on the pair (toolset_name, name). A custom tool in the same request can share a member’s name, and computer use and browser use can both expose members such as screenshot or key. Shared names are told apart by toolset_name. The two toolsets work independently, each in its own coordinate frame.

The 17 members include screenshot; zoom with region [x0, y0, x1, y1]; mouse clicks and drags; mouse_move; left_mouse_down / left_mouse_up; cursor_position; scroll; type; key; hold_key; and wait. Coordinates are in the pixel space of the full-display screenshots you return, origin at the top left. After a zoom, Claude still expresses coordinates in that full-screenshot space, never relative to the zoomed image. If you scale screenshots down before returning them, scale Claude’s coordinates back up before applying them to the real display.

All members are enabled by default, including zoom. If your environment cannot produce zoom images, withhold the member with configs rather than leaving it enabled and returning errors.

The toolset entry rejects older parameters: name, display_width_px, display_height_px, display_number, and enable_zoom. It also cannot be declared in the same request as a computer_20251124 entry or another tool named computer.

Keep the desktop on your side of the API

Claude never connects to the environment. Your application receives tool-use requests, translates them into actions, captures results such as screenshots, and returns those results. The docs describe a sandboxed setup: a virtual X11 display (Xvfb), a lightweight Linux UI with Mutter and Tint2, applications such as Firefox and LibreOffice, action handlers, and the agent loop. The reference implementation runs this inside a Docker container.

Computer use has unique risks, heightened when the environment can reach the internet. In some circumstances Claude will follow commands found in content—webpages or images—even when they conflict with your instructions. Isolate the model from sensitive data and actions. Classifiers scan tool returns such as screenshots for potential prompt injections and steer the model to check whether an instruction came from you; that extra layer is not ideal for every use case, and the isolation precautions still matter. Inform end users of relevant risks and obtain their consent before enabling computer use in your products.

The computer use tool is schema-less: you do not provide an input schema. When the toolset is present, the API generates a computer-use-specific system prompt. Your system parameter is still respected.

Implement process_tool_calls so it walks every tool_use block in order, dispatches on (toolset_name, name), and always returns a matching tool_result—including the batch halt result after the first failure. Wire that helper into sampling_loop with a hard max_iterations cap.

Next, read the computer use tool docs for member inputs, the full batch-result JSON, and prompting tips such as taking a screenshot after each step and putting instruction text before screenshot images.