Orgo - Cloud computers for AI agents

Overview

This guide shows how to get started with Anthropic’s Claude Computer Use in a couple minutes using Orgo to control a virtual desktop environment.

Setup

Install the required packages:

pip install orgo anthropic

npm install orgo @anthropic-ai/sdk

yarn add orgo @anthropic-ai/sdk

pnpm add orgo @anthropic-ai/sdk

Set up your API keys:

# Export as environment variables
export ORGO_API_KEY=your_orgo_api_key
export ANTHROPIC_API_KEY=your_anthropic_api_key

import os
os.environ["ORGO_API_KEY"] = "your_orgo_api_key"
os.environ["ANTHROPIC_API_KEY"] = "your_anthropic_api_key"

process.env.ORGO_API_KEY = "your_orgo_api_key";
process.env.ANTHROPIC_API_KEY = "your_anthropic_api_key";

Simple Usage

The simplest way to use Orgo with Claude is through the built-in prompt() method:

from orgo import Computer

# Initialize a computer
computer = Computer()

# Let Claude control the computer with natural language
computer.prompt("Open Chrome and search for pictures of cats")

# Clean up when done
computer.destroy()

import { Computer } from 'orgo';

// Initialize a computer
const computer = await Computer.create();

// Let Claude control the computer with natural language
await computer.prompt("Open Chrome and search for pictures of cats");

// Clean up when done
await computer.destroy();

This approach handles all the complexity of the agent loop automatically, making it easy to get started.

Tip: The simplest way to use Orgo is computer.prompt("your instruction"). The advanced usage below is only needed if you want to build a custom agent loop.

Customizing the Prompt Method

You can customize the prompt with optional parameters:

# Customize the model, cap the number of steps, and stream events live.
result = computer.prompt(
    "Find and download the latest Claude paper from Anthropic's website",
    model="claude-opus-4-8",   # Use Opus for complex tasks
    max_iterations=50,         # Cap the number of agent steps
    callback=lambda event, data: print(event),  # Called on each agent event
)
# `result` is the list of messages exchanged during the run.

// Customize the model, cap the number of steps, and stream events live.
const result = await computer.prompt(
    "Find and download the latest Claude paper from Anthropic's website",
    {
        model: "claude-opus-4-8",   // Use Opus for complex tasks
        maxIterations: 50,          // Cap the number of agent steps
        callback: (event, data) => console.log(event),  // Called on each agent event
    }
);
// `result` is the array of messages exchanged during the run.

Advanced Usage

For more control, you can implement your own agent loop using the Anthropic API directly:

import anthropic
from orgo import Computer

def create_agent_loop(instruction, model="claude-sonnet-5"):
    # Initialize components
    computer = Computer() 
    client = anthropic.Anthropic()
    
    try:
        # Initialize conversation
        messages = [{"role": "user", "content": instruction}]
        
        # Define tools
        tools = [
            {
                "type": "computer_20251124",  # For Claude Sonnet 4.6+
                "name": "computer",
                "display_width_px": 1024,
                "display_height_px": 768,
                "display_number": 1
            },
            {
                "type": "bash_20250124",
                "name": "bash",
            }
        ]
        
        # Start the conversation with Claude
        response = client.beta.messages.create(
            model=model,
            messages=messages,
            tools=tools,
            betas=["computer-use-2025-11-24", "computer-use-2025-01-24"],
            max_tokens=8192
        )
        
        # Add Claude's response to conversation history
        messages.append({"role": "assistant", "content": response.content})
        
        # Continue the loop until Claude stops requesting tools
        iteration = 0
        max_iterations = 20
        
        while iteration < max_iterations:
            iteration += 1
            
            # Process all tool requests from Claude
            tool_results = []
            
            for block in response.content:
                if block.type == "tool_use":
                    # Execute the requested tool action
                    result = execute_tool_action(computer, block)
                    
                    # Format the result for Claude
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": [result]
                    })
            
            # If no tools were requested, Claude is done
            if not tool_results:
                break
                
            # Send the tool results back to Claude
            messages.append({"role": "user", "content": tool_results})
            
            # Get Claude's next response
            response = client.beta.messages.create(
                model=model,
                messages=messages,
                tools=tools,
                betas=["computer-use-2025-11-24", "computer-use-2025-01-24"],
                max_tokens=8192
            )
            
            # Add Claude's response to conversation history
            messages.append({"role": "assistant", "content": response.content})
        
        return messages
        
    finally:
        # Clean up
        computer.destroy()

def execute_tool_action(computer, tool_block):
    """Execute a tool action based on Claude's request."""
    action = tool_block.input.get("action")
    
    try:
        if action == "screenshot":
            # Capture a screenshot and return as base64
            image_data = computer.screenshot_base64()
            return {
                "type": "image",
                "source": {
                    "type": "base64",
                    "media_type": "image/jpeg",
                    "data": image_data
                }
            }
            
        elif action == "left_click":
            x, y = tool_block.input["coordinate"]
            computer.left_click(x, y)
            return {"type": "text", "text": f"Clicked at ({x}, {y})"}
            
        elif action == "right_click":
            x, y = tool_block.input["coordinate"]
            computer.right_click(x, y)
            return {"type": "text", "text": f"Right-clicked at ({x}, {y})"}
            
        elif action == "double_click":
            x, y = tool_block.input["coordinate"]
            computer.double_click(x, y)
            return {"type": "text", "text": f"Double-clicked at ({x}, {y})"}
            
        elif action == "type":
            text = tool_block.input["text"]
            computer.type(text)
            return {"type": "text", "text": f"Typed: {text}"}
            
        elif action == "key":
            key = tool_block.input["text"]
            computer.key(key)
            return {"type": "text", "text": f"Pressed: {key}"}
            
        elif action == "scroll":
            direction = tool_block.input.get("scroll_direction", "down")
            amount = tool_block.input.get("scroll_amount", 1)
            computer.scroll(direction, amount)
            return {"type": "text", "text": f"Scrolled {direction} by {amount}"}
            
        elif action == "wait":
            duration = tool_block.input.get("duration", 1)
            computer.wait(duration)
            return {"type": "text", "text": f"Waited for {duration} seconds"}
            
        else:
            return {"type": "text", "text": f"Unsupported action: {action}"}
            
    except Exception as e:
        return {"type": "text", "text": f"Error executing {action}: {str(e)}"}

import { Computer } from 'orgo';
import Anthropic from '@anthropic-ai/sdk';

async function createAgentLoop(instruction: string, model = "claude-sonnet-5") {
    // Initialize components
    const computer = await Computer.create();
    const client = new Anthropic();
    
    try {
        // Initialize conversation
        const messages: any[] = [{ role: "user", content: instruction }];
        
        // Define tools
        const tools = [
            {
                type: "computer_20251124",  // For Claude Sonnet 4.6+
                name: "computer",
                display_width_px: 1024,
                display_height_px: 768,
                display_number: 1
            },
            {
                type: "bash_20250124",
                name: "bash",
            }
        ];
        
        // Start the conversation with Claude
        let response = await client.beta.messages.create({
            model,
            messages,
            tools: tools as any,
            betas: ["computer-use-2025-11-24", "computer-use-2025-01-24"],
            max_tokens: 8192
        });
        
        // Add Claude's response to conversation history
        messages.push({ role: "assistant", content: response.content });
        
        // Continue the loop until Claude stops requesting tools
        let iteration = 0;
        const maxIterations = 20;
        
        while (iteration < maxIterations) {
            iteration++;
            
            // Process all tool requests from Claude
            const toolResults = [];
            
            for (const block of response.content) {
                if (block.type === "tool_use") {
                    // Execute the requested tool action
                    const result = await executeToolAction(computer, block);
                    
                    // Format the result for Claude
                    toolResults.push({
                        type: "tool_result",
                        tool_use_id: block.id,
                        content: [result]
                    });
                }
            }
            
            // If no tools were requested, Claude is done
            if (toolResults.length === 0) {
                break;
            }
            
            // Send the tool results back to Claude
            messages.push({ role: "user", content: toolResults });
            
            // Get Claude's next response
            response = await client.beta.messages.create({
                model,
                messages,
                tools: tools as any,
                betas: ["computer-use-2025-11-24", "computer-use-2025-01-24"],
                max_tokens: 8192
            });
            
            // Add Claude's response to conversation history
            messages.push({ role: "assistant", content: response.content });
        }
        
        return messages;
        
    } finally {
        // Clean up
        await computer.destroy();
    }
}

async function executeToolAction(computer: Computer, toolBlock: any) {
    const action = toolBlock.input.action;
    
    try {
        if (action === "screenshot") {
            // Capture a screenshot and return as base64
            const imageData = await computer.screenshotBase64();
            return {
                type: "image",
                source: {
                    type: "base64",
                    media_type: "image/jpeg",
                    data: imageData
                }
            };
        } else if (action === "left_click") {
            const [x, y] = toolBlock.input.coordinate;
            await computer.leftClick(x, y);
            return { type: "text", text: `Clicked at (${x}, ${y})` };
        } else if (action === "right_click") {
            const [x, y] = toolBlock.input.coordinate;
            await computer.rightClick(x, y);
            return { type: "text", text: `Right-clicked at (${x}, ${y})` };
        } else if (action === "double_click") {
            const [x, y] = toolBlock.input.coordinate;
            await computer.doubleClick(x, y);
            return { type: "text", text: `Double-clicked at (${x}, ${y})` };
        } else if (action === "type") {
            const text = toolBlock.input.text;
            await computer.type(text);
            return { type: "text", text: `Typed: ${text}` };
        } else if (action === "key") {
            const key = toolBlock.input.text;
            await computer.key(key);
            return { type: "text", text: `Pressed: ${key}` };
        } else if (action === "scroll") {
            const direction = toolBlock.input.scroll_direction || "down";
            const amount = toolBlock.input.scroll_amount || 1;
            await computer.scroll(direction, amount);
            return { type: "text", text: `Scrolled ${direction} by ${amount}` };
        } else if (action === "wait") {
            const duration = toolBlock.input.duration || 1;
            await computer.wait(duration);
            return { type: "text", text: `Waited for ${duration} seconds` };
        } else {
            return { type: "text", text: `Unsupported action: ${action}` };
        }
    } catch (error) {
        return { type: "text", text: `Error executing ${action}: ${error}` };
    }
}

Using Claude’s Thinking Capability

Claude 4.x models can stream their reasoning process through the thinking parameter:

import anthropic
from orgo import Computer

# Initialize components
computer = Computer()
client = anthropic.Anthropic()

try:
    # Start a conversation with thinking enabled
    response = client.beta.messages.create(
        model="claude-sonnet-5",
        messages=[{"role": "user", "content": "Find an image of a cat on the web"}],
        tools=[{
            "type": "computer_20251124",
            "name": "computer",
            "display_width_px": 1024,
            "display_height_px": 768,
            "display_number": 1
        },
        {
            "type": "bash_20250124",
            "name": "bash",
        }],
        betas=["computer-use-2025-11-24", "computer-use-2025-01-24"],
        thinking={"type": "adaptive", "display": "summarized"}  # Enable thinking
    )

    # Access the thinking content
    for block in response.content:
        if block.type == "thinking":
            print("Claude's reasoning:")
            print(block.thinking)
finally:
    # Clean up
    computer.destroy()

import { Computer } from 'orgo';
import Anthropic from '@anthropic-ai/sdk';

// Initialize components
const computer = await Computer.create();
const client = new Anthropic();

try {
    // Start a conversation with thinking enabled
    const response = await client.beta.messages.create({
        model: "claude-sonnet-5",
        messages: [{ role: "user", content: "Find an image of a cat on the web" }],
        tools: [{
            type: "computer_20251124",
            name: "computer",
            display_width_px: 1024,
            display_height_px: 768,
            display_number: 1
        },
        {
            type: "bash_20250124",
            name: "bash",
        }] as any,
        betas: ["computer-use-2025-11-24", "computer-use-2025-01-24"],
        thinking: { type: "adaptive", display: "summarized" } as any  // Enable thinking
    });

    // Access the thinking content
    for (const block of response.content) {
        if (block.type === "thinking") {
            console.log("Claude's reasoning:");
            console.log((block as any).thinking);
        }
    }
} finally {
    // Clean up
    await computer.destroy();
}

Tool Compatibility

Orgo provides a complete set of methods corresponding to Claude’s computer use tools:

Claude Tool Action	Orgo Method (Python)	Orgo Method (TypeScript)	Description
`screenshot`	`computer.screenshot()`	`await computer.screenshot()`	Capture the screen (returns PIL Image/Buffer)
`screenshot`	`computer.screenshot_base64()`	`await computer.screenshotBase64()`	Capture the screen (returns base64 string)
`left_click`	`computer.left_click(x, y)`	`await computer.leftClick(x, y)`	Left click at coordinates
`right_click`	`computer.right_click(x, y)`	`await computer.rightClick(x, y)`	Right click at coordinates
`double_click`	`computer.double_click(x, y)`	`await computer.doubleClick(x, y)`	Double click at coordinates
`type`	`computer.type(text)`	`await computer.type(text)`	Type text
`key`	`computer.key(key_sequence)`	`await computer.key(keySequence)`	Press keys (e.g., “Enter”, “ctrl+c”)
`scroll`	`computer.scroll(direction, amount)`	`await computer.scroll(direction, amount)`	Scroll in specified direction
`wait`	`computer.wait(seconds)`	`await computer.wait(seconds)`	Wait for specified seconds

Picking a model

Model	When to use
`claude-opus-4-8`	Hardest, multi-step desktop tasks where accuracy and judgment matter most.
`claude-sonnet-5`	The default for most computer-use agents - fast, capable, and dramatically cheaper than Opus.
`claude-haiku-4-5`	Tight latency budgets, simple flows, or high-volume parallel agents.

All three are computer-use capable and share the same tool surface.

Tool versions

Match the tool type and betas to the model family you’re calling:

Claude 4.x (Opus 4.7 / Sonnet 4.6 / Haiku 4.5): "type": "computer_20251124" with betas ["computer-use-2025-11-24", "computer-use-2025-01-24"]
Claude Sonnet 4.5: "type": "computer_20250124" with betas ["computer-use-2025-01-24"]

TypeScript users: All methods are async and must be awaited. The TypeScript SDK uses camelCase for method names (e.g., leftClick instead of left_click).

Video Tutorial

Here is a video version showing how to set up Claude Computer Use in 30 seconds:

You can follow the video tutorial above or use this written guide

Get Started

Guides

Migrations

Templates

Model integrations

Claude Computer Use

Overview

Setup

Simple Usage

Customizing the Prompt Method

Advanced Usage

Using Claude’s Thinking Capability

Tool Compatibility

Picking a model

Tool versions

Video Tutorial

​Overview

​Setup

​Simple Usage

​Customizing the Prompt Method

​Advanced Usage

​Using Claude’s Thinking Capability

​Tool Compatibility

​Picking a model

​Tool versions

​Video Tutorial

Overview

Setup

Simple Usage

Customizing the Prompt Method

Advanced Usage

Using Claude’s Thinking Capability

Tool Compatibility

Picking a model

Tool versions

Video Tutorial