Vercel Labs ships agent-desktop for coding agents

Vercel Labs has shipped a desktop tool for coding agents, and it is not itself an agent.

On October 31, 2026, Chris Tate introduced agent-desktop as “agent-browser, for your whole computer”: computer use for any coding agent, on macOS, Windows, and Linux. He said it runs in the background while you keep working, and that agents can use the computer in parallel, each with its own cursor. Minutes later, Malte Ubl posted “New from Vercel Labs: agent-desktop,” writing that it is “the same token-efficient computer use you know and love from agent-browser but for the entire desktop operating system.” The GitHub README marks the project as a Vercel Labs product. The license is Apache-2.0.

The docs are explicit about what shipped. agent-desktop is a tool that agents call, not an agent. It reads a desktop app’s accessibility tree and returns JSON. Every element the caller can act on gets a ref. The agent acts on that ref, reads the UI again, and repeats. There is no model inside: it never embeds an LLM, has no GUI, and has no interactive prompt. Every command prints one JSON envelope and exits. The observe, act, and verify loop stays in the calling agent or script. A C library, libagent_desktop_ffi, can do the same work in process instead of starting the CLI for each command.

That is a different picture from a cursor on screen. Tate’s parallel-cursor line is the announcement. The README’s default is headless. Ref actions go through the accessibility API without moving the pointer or taking focus. The global --headed flag is what allows physical input. Unless that flag is on, the published path is structured tree actions, not pointer movement.

Two other limits are easy to miss. It does not automate web pages; agent-browser remains the tool for those. For the web contents of a Chromium or Electron app you launch, agent-desktop can open a DevTools port that agent-browser connects to. It also does not match pixels, so it cannot see a UI that publishes no accessibility tree.

Under the CLI, agent-desktop-core owns commands, snapshot building, and a RefStore. Core reaches an operating system only through a PlatformAdapter. One platform crate is chosen at compile time: agent-desktop-macos, agent-desktop-windows, or agent-desktop-linux. Each crate calls that operating system’s accessibility API — the AX API on macOS, UI Automation on Windows, or AT-SPI on Linux. Core never imports a platform crate, and the platform crates never import each other, so a binary drives the OS it was built for.

The CLI has the same 60 command names on every platform. Fifty-six are operational. key-down, key-up, mouse-down, and mouse-up fail closed; the README says they refuse to run until a daemon can release a held key or button. The names are shared; the surfaces are not. Linux mouse input is listed as X11 and --headed. Wayland clipboard needs wl-clipboard. open-system-surface is marked planned on macOS, supported on Windows, and listed for the GNOME tray and Action Center on Linux.

Install is npm install -g agent-desktop on macOS, Windows, and Linux. Bun, Yarn, and pnpm can install the same package, which includes native binaries and platform helpers for x64 and ARM64. For a binary without a JavaScript launcher, the README points to the matching native archive on GitHub Releases and says to keep its helper beside the executable. Official Homebrew and Scoop packages are being submitted and are not yet in the default catalogs. The repository no longer maintains a custom Homebrew tap or Scoop bucket.

The OS gates are specific. macOS 13 or later, Intel or Apple Silicon, needs Accessibility permission for the app that runs agent-desktop. Windows needs Windows 10 1809 or later, or Windows Server 2019 or later, on x64 or ARM64, in the signed-in user’s interactive desktop. The README says Windows needs no extra grant for apps at the terminal’s integrity level. Linux needs Ubuntu 24.04 or later on x86_64 or aarch64, with a glibc 2.35 floor, in the user’s graphical session and a running AT-SPI bus. The README names at-spi2-core as the Linux requirement.

Refs belong to the snapshot that produced them. A bare ref is rejected with INVALID_ARGS. When the UI changes, an action can return STALE_REF or AMBIGUOUS_TARGET; take a new snapshot and use its refs. On macOS, CLI calls reuse a private command host for each snapshot namespace and binary build, and retain native accessibility objects so a removed row cannot silently resolve to a surviving row with the same identifier. The host exits after 15 idle minutes. On macOS, headless calls still prohibit foreground activation and synthesized input.

Snapshots, clipboard reads, screenshots, command output, and debug artifacts can contain private desktop content. The README says to keep them out of public logs and uploads. An optional JEV helper can send screen descriptions to a configured model provider; it is separate from the native CLI.

Start with the installation page, then the vercel-labs/agent-desktop GitHub repo. Have the agent run agent-desktop skills get desktop and agent-desktop skills get platform so it loads the loop, ref, and OS rules that match the binary it will actually call.

Sources