Orgo - Cloud computers for AI agents

Wire Anthropic’s computer use toolset to an Orgo computer. Claude sees a screenshot, asks for a click or a keystroke, and your loop runs it on the computer.

Coordinate space

Orgo computers boot at 1280x720x24. The current toolset, computer_toolset_20260801, takes no display size. Anthropic’s documentation states the rule directly: coordinates are always in the pixel space of the screenshots you return. Send an Orgo screenshot unresized and the coordinates Claude returns are already the computer’s pixels. Pass them straight to left_click. A 1280x720 screenshot is inside Claude’s image size limit, so there is no reason to shrink one. If you resize anyway, keep the scale factor and divide the coordinates back before you click:

# Only needed if you resized the screenshot before sending it.
def to_computer_pixels(x, y, scale):
    return round(x / scale), round(y / scale)

Send a downscaled screenshot and click the raw coordinates and every click lands short of where Claude aimed. Nothing errors. The agent looks like it is confidently clicking the wrong thing.

The older tool versions work differently. computer_20251124 and computer_20250124 still take display_width_px and display_height_px, and those must match the size of the screenshots you send. On a default Orgo computer that is 1280 and 720. If you create a computer at a non-default resolution, or resize its screen later, use that resolution instead. GET /computers/{id}/screens reports the current width and height.

Setup

Install the required packages:

pip install orgo anthropic

npm install orgo @anthropic-ai/sdk

yarn add orgo @anthropic-ai/sdk

pnpm add orgo @anthropic-ai/sdk

Set up your API keys:

# Export as environment variables
export ORGO_API_KEY=your_orgo_api_key
export ANTHROPIC_API_KEY=your_anthropic_api_key

import os
os.environ["ORGO_API_KEY"] = "your_orgo_api_key"
os.environ["ANTHROPIC_API_KEY"] = "your_anthropic_api_key"

process.env.ORGO_API_KEY = "your_orgo_api_key";
process.env.ANTHROPIC_API_KEY = "your_anthropic_api_key";

Simple usage

The quickest way to use Orgo with Claude is the built-in prompt() method:

from orgo import Computer

# Initialize a computer
computer = Computer()

# Let Claude control the computer with natural language
computer.prompt("Open Chrome and search for pictures of cats")

# Clean up when done
computer.destroy()

import { Computer } from 'orgo';

// Initialize a computer
const computer = await Computer.create();

// Let Claude control the computer with natural language
await computer.prompt("Open Chrome and search for pictures of cats");

// Clean up when done
await computer.destroy();

This runs the whole agent loop for you.

Tip: computer.prompt("your instruction") is all most projects need. Reach for the advanced usage below only when you want to own the loop.

Customizing the prompt method

You can customize the prompt with optional parameters:

# Customize the model, cap the number of steps, and stream events live.
result = computer.prompt(
    "Find and download the latest Claude paper from Anthropic's website",
    model="claude-opus-5",     # Use Opus for complex tasks
    max_iterations=50,         # Cap the number of agent steps
    callback=lambda event, data: print(event),  # Called on each agent event
)
# `result` is the list of messages exchanged during the run.

// Customize the model, cap the number of steps, and stream events live.
const result = await computer.prompt(
    "Find and download the latest Claude paper from Anthropic's website",
    {
        model: "claude-opus-5",     // Use Opus for complex tasks
        maxIterations: 50,          // Cap the number of agent steps
        callback: (event, data) => console.log(event),  // Called on each agent event
    }
);
// `result` is the array of messages exchanged during the run.

Advanced usage

For full control, drive the Anthropic API yourself. Three things about the toolset shape the loop:

One entry in tools declares every member. The member name arrives in the tool_use block’s name, and there is no action field in input.
Claude can return several member calls in one turn. Run them in order and stop at the first failure.
Return one tool_result per tool_use block, in a single user message. Member results echo toolset_name.

import anthropic
from orgo import Computer

TOOLS = [
    {"type": "computer_toolset_20260801"},
    {"type": "bash_20250124", "name": "bash"},
]


def create_agent_loop(instruction, model="claude-sonnet-5", max_iterations=20):
    computer = Computer()
    client = anthropic.Anthropic()

    try:
        messages = [{"role": "user", "content": instruction}]

        for _ in range(max_iterations):
            response = client.messages.create(
                model=model,
                messages=messages,
                tools=TOOLS,
                max_tokens=8192,
            )
            messages.append({"role": "assistant", "content": response.content})

            # No tool calls means Claude is done.
            tool_results = run_tool_calls(computer, response)
            if not tool_results:
                break

            messages.append({"role": "user", "content": tool_results})

        return messages

    finally:
        computer.destroy()


def run_tool_calls(computer, response):
    """Run one turn's tool calls in order, stopping at the first failure."""
    results = []
    failed = False

    for block in response.content:
        if block.type != "tool_use":
            continue

        if block.name == "bash":
            results.append({
                "type": "tool_result",
                "tool_use_id": block.id,
                "content": computer.bash(block.input["command"]),
            })
            continue

        # Dispatch on the pair, because a custom tool may reuse a member name.
        if getattr(block, "toolset_name", None) != "computer":
            continue

        result = {
            "type": "tool_result",
            "tool_use_id": block.id,
            "toolset_name": "computer",
        }

        if failed:
            result["content"] = "Not executed: an earlier computer action in this turn failed."
            result["is_error"] = True
        else:
            try:
                result["content"] = run_member(computer, block.name, block.input)
            except Exception as error:
                result["content"] = f"Error: {error}"
                result["is_error"] = True
                failed = True

        results.append(result)

    return results


def run_member(computer, name, params):
    """Map one member tool to an Orgo call. Only screenshot returns an image."""
    if name == "screenshot":
        return [{
            "type": "image",
            "source": {
                "type": "base64",
                "media_type": "image/jpeg",
                "data": computer.screenshot_base64(),
            },
        }]

    if name == "left_click":
        computer.left_click(*params["coordinate"])
    elif name == "right_click":
        computer.right_click(*params["coordinate"])
    elif name == "double_click":
        computer.double_click(*params["coordinate"])
    elif name == "type":
        computer.type(params["text"])
    elif name == "key":
        computer.key(params["text"])
    elif name == "scroll":
        computer.scroll(params["scroll_direction"], params["scroll_amount"])
    elif name == "wait":
        computer.wait(params["duration"])
    else:
        raise ValueError(f"Unsupported member: {name}")

    return "OK"

import { Computer } from 'orgo';
import Anthropic from '@anthropic-ai/sdk';

const TOOLS: any[] = [
    { type: "computer_toolset_20260801" },
    { type: "bash_20250124", name: "bash" },
];

async function createAgentLoop(
    instruction: string,
    model = "claude-sonnet-5",
    maxIterations = 20,
) {
    const computer = await Computer.create();
    const client = new Anthropic();

    try {
        const messages: any[] = [{ role: "user", content: instruction }];

        for (let i = 0; i < maxIterations; i++) {
            const response = await client.messages.create({
                model,
                messages,
                tools: TOOLS,
                max_tokens: 8192,
            });
            messages.push({ role: "assistant", content: response.content });

            // No tool calls means Claude is done.
            const toolResults = await runToolCalls(computer, response);
            if (toolResults.length === 0) break;

            messages.push({ role: "user", content: toolResults });
        }

        return messages;

    } finally {
        await computer.destroy();
    }
}

async function runToolCalls(computer: Computer, response: any) {
    const results: any[] = [];
    let failed = false;

    for (const block of response.content) {
        if (block.type !== "tool_use") continue;

        if (block.name === "bash") {
            results.push({
                type: "tool_result",
                tool_use_id: block.id,
                content: await computer.bash(block.input.command),
            });
            continue;
        }

        // Dispatch on the pair, because a custom tool may reuse a member name.
        if (block.toolset_name !== "computer") continue;

        const result: any = {
            type: "tool_result",
            tool_use_id: block.id,
            toolset_name: "computer",
        };

        if (failed) {
            result.content = "Not executed: an earlier computer action in this turn failed.";
            result.is_error = true;
        } else {
            try {
                result.content = await runMember(computer, block.name, block.input);
            } catch (error) {
                result.content = `Error: ${error}`;
                result.is_error = true;
                failed = true;
            }
        }

        results.push(result);
    }

    return results;
}

async function runMember(computer: Computer, name: string, params: any) {
    if (name === "screenshot") {
        return [{
            type: "image",
            source: {
                type: "base64",
                media_type: "image/jpeg",
                data: await computer.screenshotBase64(),
            },
        }];
    }

    const [x, y] = params.coordinate ?? [];

    switch (name) {
        case "left_click":   await computer.leftClick(x, y); break;
        case "right_click":  await computer.rightClick(x, y); break;
        case "double_click": await computer.doubleClick(x, y); break;
        case "type":         await computer.type(params.text); break;
        case "key":          await computer.key(params.text); break;
        case "scroll":       await computer.scroll(params.scroll_direction, params.scroll_amount); break;
        case "wait":         await computer.wait(params.duration); break;
        default: throw new Error(`Unsupported member: ${name}`);
    }

    return "OK";
}

The toolset has 17 members, and the loop above maps the 8 Orgo has a method for. The rest come back as an error tool_result, which Claude recovers from but wastes a turn on. Disable them with configs so Claude never sees them:

{
    "type": "computer_toolset_20260801",
    "configs": {
        name: {"enabled": False}
        for name in [
            "zoom", "middle_click", "triple_click", "left_click_drag",
            "mouse_move", "left_mouse_down", "left_mouse_up",
            "cursor_position", "hold_key",
        ]
    },
}

A configs that disables every member is rejected, so drop the whole entry instead of emptying it.

Using Claude’s thinking capability

Claude can stream its reasoning through the thinking parameter:

import anthropic
from orgo import Computer

# Initialize components
computer = Computer()
client = anthropic.Anthropic()

try:
    response = client.messages.create(
        model="claude-sonnet-5",
        max_tokens=8192,
        messages=[{"role": "user", "content": "Find an image of a cat on the web"}],
        tools=[
            {"type": "computer_toolset_20260801"},
            {"type": "bash_20250124", "name": "bash"},
        ],
        thinking={"type": "adaptive", "display": "summarized"},
    )

    # Access the thinking content
    for block in response.content:
        if block.type == "thinking":
            print("Claude's reasoning:")
            print(block.thinking)
finally:
    # Clean up
    computer.destroy()

import { Computer } from 'orgo';
import Anthropic from '@anthropic-ai/sdk';

// Initialize components
const computer = await Computer.create();
const client = new Anthropic();

try {
    const response = await client.messages.create({
        model: "claude-sonnet-5",
        max_tokens: 8192,
        messages: [{ role: "user", content: "Find an image of a cat on the web" }],
        tools: [
            { type: "computer_toolset_20260801" },
            { type: "bash_20250124", name: "bash" },
        ] as any,
        thinking: { type: "adaptive", display: "summarized" },
    });

    // Access the thinking content
    for (const block of response.content) {
        if (block.type === "thinking") {
            console.log("Claude's reasoning:");
            console.log((block as any).thinking);
        }
    }
} finally {
    // Clean up
    await computer.destroy();
}

Set display: "summarized" to read the reasoning. The current models default to "omitted", which streams thinking blocks with empty text.

Tool compatibility

Orgo has a method for each member tool a desktop agent reaches for:

Toolset member	Orgo method (Python)	Orgo method (TypeScript)	Description
`screenshot`	`computer.screenshot()`	`await computer.screenshot()`	Capture the screen (returns PIL Image/Buffer)
`screenshot`	`computer.screenshot_base64()`	`await computer.screenshotBase64()`	Capture the screen (returns base64 JPEG)
`left_click`	`computer.left_click(x, y)`	`await computer.leftClick(x, y)`	Left click at coordinates
`right_click`	`computer.right_click(x, y)`	`await computer.rightClick(x, y)`	Right click at coordinates
`double_click`	`computer.double_click(x, y)`	`await computer.doubleClick(x, y)`	Double click at coordinates
`type`	`computer.type(text)`	`await computer.type(text)`	Type text
`key`	`computer.key(key_sequence)`	`await computer.key(keySequence)`	Press keys (e.g. “Return”, “ctrl+c”)
`scroll`	`computer.scroll(direction, amount)`	`await computer.scroll(direction, amount)`	Scroll in a direction
`wait`	`computer.wait(seconds)`	`await computer.wait(seconds)`	Wait for a number of seconds

Only screenshot and zoom return images. Every other member returns text, so "OK" is a valid result.

Picking a model

Model	When to use
`claude-opus-5`	Hardest, multi-step desktop tasks where accuracy and judgment matter most.
`claude-sonnet-5`	The default for most computer-use agents: fast, capable, and cheaper than Opus.
`claude-opus-4-8`	The previous Opus generation, still supported on the current toolset.

claude-fable-5 also supports the toolset. claude-haiku-4-5 does not support computer use.

Tool versions

Match the tool type to the model you are calling:

Tool `type`	Models	Beta header
`computer_toolset_20260801`	Opus 5, Sonnet 5, Opus 4.8, Fable 5	None
`computer_20251124`	Opus 4.7, Opus 4.6, Sonnet 4.6, Opus 4.5	`computer-use-2025-11-24`
`computer_20250124`	Earlier Claude models	`computer-use-2025-01-24`

The two dated tool versions remain available for existing integrations. Use the toolset for anything new: it needs no beta header, it batches several actions into one turn, and it drops the display size fields that were the most common source of misplaced clicks. The bash_20250124 tool needs no beta header on any of them.

TypeScript users: All methods are async and must be awaited. The TypeScript SDK uses camelCase for method names (e.g. leftClick instead of left_click).

Video tutorial

The video covers setup. The tool shapes above are the current ones.

Get Started

Guides

Migrations

Templates

Model integrations

Claude Computer Use

Coordinate space

Setup

Simple usage

Customizing the prompt method

Advanced usage

Using Claude’s thinking capability

Tool compatibility

Picking a model

Tool versions

Video tutorial

​Coordinate space

​Setup

​Simple usage

​Customizing the prompt method

​Advanced usage

​Using Claude’s thinking capability

​Tool compatibility

​Picking a model

​Tool versions

​Video tutorial

Coordinate space

Setup

Simple usage

Customizing the prompt method

Advanced usage

Using Claude’s thinking capability

Tool compatibility

Picking a model

Tool versions

Video tutorial