Skip to main content

​
Overview

This guide shows how to get started with Anthropic’s Claude Computer Use in a couple minutes using Orgo to control a virtual desktop environment.

​
Setup

Install the required packages: 
pip install orgo anthropic
Set up your API keys: 
# Export as environment variables
export ORGO_API_KEY=your_orgo_api_key
export ANTHROPIC_API_KEY=your_anthropic_api_key

​
Simple Usage

The simplest way to use Orgo with Claude is through the built-in prompt() method: 
from orgo import Computer

# Initialize a computer
computer = Computer()

# Let Claude control the computer with natural language
computer.prompt("Open Firefox and search for pictures of cats")

# Clean up when done
computer.destroy()
This approach handles all the complexity of the agent loop automatically, making it easy to get started.

​
Customizing the Prompt Method

You can customize the prompt experience with various parameters: 
# Create a progress callback
def progress_callback(event_type, event_data):
    if event_type == "text":
        print(f"Claude: {event_data}")
    elif event_type == "tool_use":
        print(f"Action: {event_data['action']}")
    elif event_type == "thinking":
        print(f"Thinking: {event_data}")

# Use Claude with custom parameters
messages = computer.prompt(
    instruction="Find and download the latest Claude paper from Anthropic's website",
    model="claude-sonnet-4-20250514",  # The model to use
    display_width=1280,  # Set screen resolution
    display_height=800,
    callback=progress_callback,  # Track progress
    thinking_enabled=True,  # Enable Claude's "thinking" capability (Claude 3.7+)
    max_iterations=15,  # Limit the number of agent loops
    max_tokens=4096,  # Maximum tokens for Claude responses
    api_key="your_anthropic_api_key"  # Override environment variable
)

​
Advanced Usage

For more control, you can implement your own agent loop using the Anthropic API directly: 
import anthropic
from orgo import Computer

def create_agent_loop(instruction, model="claude-sonnet-4-20250514"):
    # Initialize components
    computer = Computer() 
    client = anthropic.Anthropic()
    
    try:
        # Initialize conversation
        messages = [{"role": "user", "content": instruction}]
        
        # Define tools
        tools = [
            {
                "type": "computer_20250124",  # For Claude 3.7+
                "name": "computer",
                "display_width_px": 1024,
                "display_height_px": 768,
                "display_number": 1
            }
        ]
        
        # Start the conversation with Claude
        response = client.beta.messages.create(
            model=model,
            messages=messages,
            tools=tools,
            betas=["computer-use-2025-01-24"],
            max_tokens=4096
        )
        
        # Add Claude's response to conversation history
        messages.append({"role": "assistant", "content": response.content})
        
        # Continue the loop until Claude stops requesting tools
        iteration = 0
        max_iterations = 20
        
        while iteration < max_iterations:
            iteration += 1
            
            # Process all tool requests from Claude
            tool_results = []
            
            for block in response.content:
                if block.type == "tool_use":
                    # Execute the requested tool action
                    result = execute_tool_action(computer, block)
                    
                    # Format the result for Claude
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": [result]
                    })
            
            # If no tools were requested, Claude is done
            if not tool_results:
                break
                
            # Send the tool results back to Claude
            messages.append({"role": "user", "content": tool_results})
            
            # Get Claude's next response
            response = client.beta.messages.create(
                model=model,
                messages=messages,
                tools=tools,
                betas=["computer-use-2025-01-24"],
                max_tokens=4096
            )
            
            # Add Claude's response to conversation history
            messages.append({"role": "assistant", "content": response.content})
        
        return messages
        
    finally:
        # Clean up
        computer.destroy()

def execute_tool_action(computer, tool_block):
    """Execute a tool action based on Claude's request."""
    action = tool_block.input.get("action")
    
    try:
        if action == "screenshot":
            # Capture a screenshot and return as base64
            image_data = computer.screenshot_base64()
            return {
                "type": "image",
                "source": {
                    "type": "base64",
                    "media_type": "image/jpeg",
                    "data": image_data
                }
            }
            
        elif action == "left_click":
            x, y = tool_block.input["coordinate"]
            computer.left_click(x, y)
            return {"type": "text", "text": f"Clicked at ({x}, {y})"}
            
        elif action == "right_click":
            x, y = tool_block.input["coordinate"]
            computer.right_click(x, y)
            return {"type": "text", "text": f"Right-clicked at ({x}, {y})"}
            
        elif action == "double_click":
            x, y = tool_block.input["coordinate"]
            computer.double_click(x, y)
            return {"type": "text", "text": f"Double-clicked at ({x}, {y})"}
            
        elif action == "type":
            text = tool_block.input["text"]
            computer.type(text)
            return {"type": "text", "text": f"Typed: {text}"}
            
        elif action == "key":
            key = tool_block.input["text"]
            computer.key(key)
            return {"type": "text", "text": f"Pressed: {key}"}
            
        elif action == "scroll":
            direction = tool_block.input.get("scroll_direction", "down")
            amount = tool_block.input.get("scroll_amount", 1)
            computer.scroll(direction, amount)
            return {"type": "text", "text": f"Scrolled {direction} by {amount}"}
            
        elif action == "wait":
            duration = tool_block.input.get("duration", 1)
            computer.wait(duration)
            return {"type": "text", "text": f"Waited for {duration} seconds"}
            
        else:
            return {"type": "text", "text": f"Unsupported action: {action}"}
            
    except Exception as e:
        return {"type": "text", "text": f"Error executing {action}: {str(e)}"}

​
Using Claude’s Thinking Capability

Claude 4 Sonnet can provide its reasoning process through the thinking parameter: 
import anthropic
from orgo import Computer

# Initialize components
computer = Computer()
client = anthropic.Anthropic()

try:
    # Start a conversation with thinking enabled
    response = client.beta.messages.create(
        model="claude-sonnet-4-20250514",
        messages=[{"role": "user", "content": "Find an image of a cat on the web"}],
        tools=[{
            "type": "computer_20250124",
            "name": "computer",
            "display_width_px": 1024,
            "display_height_px": 768,
            "display_number": 1
        }],
        betas=["computer-use-2025-01-24"],
        thinking={"type": "enabled", "budget_tokens": 1024}  # Enable thinking
    )

    # Access the thinking content
    for block in response.content:
        if block.type == "thinking":
            print("Claude's reasoning:")
            print(block.thinking)
finally:
    # Clean up
    computer.destroy()

​
Tool Compatibility

Orgo provides a complete set of methods corresponding to Claude’s computer use tools:
Claude Tool ActionOrgo Method (Python)Orgo Method (TypeScript)Description
screenshotcomputer.screenshot()await computer.screenshot()Capture the screen (returns PIL Image/Buffer)
screenshotcomputer.screenshot_base64()await computer.screenshotBase64()Capture the screen (returns base64 string)
left_clickcomputer.left_click(x, y)await computer.leftClick(x, y)Left click at coordinates
right_clickcomputer.right_click(x, y)await computer.rightClick(x, y)Right click at coordinates
double_clickcomputer.double_click(x, y)await computer.doubleClick(x, y)Double click at coordinates
typecomputer.type(text)await computer.type(text)Type text
keycomputer.key(key_sequence)await computer.key(keySequence)Press keys (e.g., “Enter”, “ctrl+c”)
scrollcomputer.scroll(direction, amount)await computer.scroll(direction, amount)Scroll in specified direction
waitcomputer.wait(seconds)await computer.wait(seconds)Wait for specified seconds

​
Claude 4 vs 3.5 Sonnet

When using different Claude models, make sure to use the appropriate tool type:
  • For Claude 4 Sonnet: "type": "computer_20250124"
  • For Claude 3.5 Sonnet: "type": "computer_20241022"
And use the corresponding beta flag:
  • For Claude 4 Sonnet: betas=["computer-use-2025-01-24"]
  • For Claude 3.5 Sonnet: betas=["computer-use-2024-10-22"]
TypeScript users: All methods are async and must be awaited. The TypeScript SDK uses camelCase for method names (e.g., leftClick instead of left_click).

​
Video Tutorial

Here is a video version showing how to set up Claude Computer Use in 30 seconds: You can follow the video tutorial above or use this written guide
⌘I