computer-use

Computer Use Automation (TypeScript)

AI-powered browser automation using computer use capabilities from multiple providers with Intuned's infrastructure.

Run on Intuned

Overview

This project demonstrates four different approaches to AI-powered browser automation:

1. Anthropic Computer Use

Uses Claude's native computer use API with X/Y coordinate-based interactions and extended thinking capabilities.

2. OpenAI Computer Use

Uses OpenAI's gpt-5.4 Responses API computer-use flow for autonomous browser control with X/Y coordinate-based interactions.

3. Gemini Computer Use

Uses Google's Gemini with Stagehand for computer use automation, leveraging Gemini's CUA capabilities with Stagehand's agent framework.

4. Stagehand (Non-CUA)

Stagehand agent without computer use API for traditional DOM-based automation.

APIs

Anthropic Computer Use (api/anthropic-computer-use.ts)

Claude-powered automation using native computer use tools.

Parameters:

{
  query: string;  // The task you want the AI to perform
}

OpenAI Computer Use (api/openai-computer-use.ts)

OpenAI's gpt-5.4 computer-use model path for autonomous browser control.

Parameters:

{
  query: string;  // The task you want the AI to perform
}

Gemini Computer Use (api/gemini-computer-use.ts)

Google's Gemini with Stagehand for enhanced computer use automation.

Parameters:

{
  query: string;  // The task you want the AI to perform
}

Stagehand (api/stagehand.ts)

Stagehand agent without computer use API for traditional DOM-based automation.

Parameters:

{
  query: string;  // The task you want the AI to perform
}

How It Works

Anthropic & OpenAI Implementations

Both use native computer use APIs that operate with X/Y coordinates to control the browser, mimicking human computer interaction. They capture screenshots, decide on actions, and execute mouse/keyboard commands.

Gemini Implementation

Uses Stagehand with Gemini's computer use capabilities, combining CUA (Computer Use API) with Stagehand's agent framework for enhanced automation.

Stagehand Implementation

Uses the Stagehand library which provides DOM-based automation with built-in tools for common browser tasks.

Getting started

Install dependencies

npm install
# or
yarn

If the intuned CLI is not installed, install it globally:

npm install -g @intuned/cli

After installing dependencies, intuned command should be available in your environment.

Environment Variables

The Gemini Computer Use implementation requires GEMINI_API_KEY:

export GEMINI_API_KEY=your_api_key_here

Run an API

# Anthropic Computer Use
intuned dev run api anthropic-computer-use .parameters/api/anthropic-computer-use/default.json

# OpenAI Computer Use
intuned dev run api openai-computer-use .parameters/api/openai-computer-use/default.json

# Gemini Computer Use
intuned dev run api gemini-computer-use .parameters/api/gemini-computer-use/default.json

# Stagehand
intuned dev run api stagehand .parameters/api/stagehand/default.json

Save project

intuned dev provision

Deploy

intuned dev deploy

Project Structure

/
├── api/
│   ├── anthropic-computer-use.ts   # Anthropic Claude computer use
│   ├── openai-computer-use.ts      # OpenAI computer use
│   ├── gemini-computer-use.ts      # Gemini with Stagehand
│   └── stagehand.ts                # Stagehand without CUA
├── hooks/
│   └── setupContext.ts             # Setup hook for Stagehand and Gemini
├── .parameters/api/                # Parameter files for each API
│   ├── anthropic-computer-use/
│   │   └── default.json
│   ├── openai-computer-use/
│   │   └── default.json
│   ├── gemini-computer-use/
│   │   └── default.json
│   └── stagehand/
│       └── default.json
├── lib/                            # Shared libraries for Anthropic and OpenAI
│   ├── anthropic/                  # Anthropic computer use implementation
│   └── openai/                     # OpenAI computer use implementation
├── intuned-resources/
│   └── jobs/
│       ├── anthropic-computer-use.job.jsonc  # Job for Anthropic computer use
│       ├── openai-computer-use.job.jsonc     # Job for OpenAI computer use
│       ├── gemini-computer-use.job.jsonc     # Job for Gemini computer use
│       └── stagehand.job.jsonc               # Job for Stagehand
├── package.json                    # Dependencies
├── tsconfig.json                   # TypeScript configuration
└── Intuned.jsonc                   # Intuned configuration

Related

• Intuned CLI
• Intuned Browser SDK
• Anthropic Computer Use: https://docs.anthropic.com/en/docs/computer-use
• OpenAI Computer Use: https://developers.openai.com/api/docs/guides/tools-computer-use
• Stagehand Documentation: https://docs.stagehand.dev/
• Intuned llm.txt

Acknowledgments

The Anthropic and OpenAI implementations were built using templates from create-kernel-app.