Experimental, pre-release
$ mobile-agent
Drives Android devices and emulators, iOS Simulators and iPhones for AI agents: a daemon with a live mirror of every screen, a CLI and an MCP server, built around the latency of each tap-and-observe step.
A command sequence written for this task in the form an agent issues it, replayed by a script through the CLI, on an iPhone 17 Simulator. Timings as the CLI printed them.
- $ mobile-agent open com.apple.mobilesafari✓ Opened com.apple.mobilesafari on udid redacted
- $ mobile-agent click '#TabBarItemTitle'✓ Done (hit text "Search or enter website" (no ref), dispatch 0.0 ms, first frame change 457.2 ms, first tree delta 541.4 ms)
- $ mobile-agent type mobileagent.dev✓ Done (read back "mobileagent.dev", typed)
- $ mobile-agent press Enter✓ Done
- $ mobile-agent wait --settle✓ Settled after 1436.71 ms: the tree changed last
- $ mobile-agent scroll down 2400✓ Done (4 strokes, at rest 2747.8 ms after the last)
- $ mobile-agent wait --settle✓ Settled after 194.06 ms: the tree changed last
A command sequence written for this task in the form an agent issues it, replayed by a script through the CLI, on an iPhone 17 Simulator. Timings as the CLI printed them.
What it is
Experimental and pre-release: built from source, with the drivers on private Apple and hidden Android interfaces, so it is development tooling, never something to ship in an app. Status and limitations
A daemon with a live mirror
It keeps each attached screen's windows, accessibility tree and latest frame current from the platform's change events, and answers snapshots, waits and settle checks from that mirror instead of reading the device on every request.
docs/overview.mdThe CLI
mobile-agent follows agent-browser's command grammar, @eN refs and JSON envelope, one process per command, so prompts and skills written for agent-browser carry over.
Agent guideThe MCP server
mobile-agent-mcp gives Claude Code, Claude Desktop, Codex or Cursor the same tools, refs and replies over the Model Context Protocol, on the same daemon.
Agent surfacesWhat it drives
Android emulators and devices through an agent pushed over adb, booted iOS Simulators from the host with nothing installed, and physical iPhones through a runner built in the repository. Built from source on macOS; Linux builds the Android driver.
Supported platformsAgainst the baselines
The same reference flows on the repository's fixtures, run by its harness through each tool as an agent would, one process per command. mobile-agent settles and verifies after every step.
iPhone 17 Simulator, iOS 27.0, Xcode 27.1 · 10 measured iterations after 2 warm-up · CI run of 2026-10-31
loginsign in on the fixture's login screen and wait for the welcome · p50 seconds, lower is better
login on iOS Simulator as a table
| Fixture | Tool | p50 (ms) | Factor | Failed / flaky | Source |
|---|---|---|---|---|---|
| SwiftUI | mobile-agent 0.1.0 | 7078.374 | 0% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact | |
| SwiftUI | Maestro 2.10.0 | 35899.719 | 5.07× | 0% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact |
| SwiftUI | agent-device 0.21.14 | 12989.613 | 1.84× | 0% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact |
list-scrollscroll a long list to row 200 and open its detail · p50 seconds, lower is better
list-scroll on iOS Simulator as a table
| Fixture | Tool | p50 (ms) | Factor | Failed / flaky | Source |
|---|---|---|---|---|---|
| SwiftUI | mobile-agent 0.1.0 | not timed: golden check failed: the content stopped moving before row-200 was on screen | 100% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact | |
| SwiftUI | Maestro 2.10.0 | not timed: golden check failed: timed out after 120 s | 100% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact | |
| SwiftUI | agent-device 0.21.14 | not timed: golden check failed: selector id="tab-list" did not match, then its scroll spent its 12-pass budget before row-200 | 100% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact |
big-screenopen the 500-cell screen and read it · p50 seconds, lower is better
big-screen on iOS Simulator as a table
| Fixture | Tool | p50 (ms) | Factor | Failed / flaky | Source |
|---|---|---|---|---|---|
| SwiftUI | mobile-agent 0.1.0 | 4818.824 | 0% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact | |
| SwiftUI | Maestro 2.10.0 | 30038.028 | 6.23× | 0% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact |
| SwiftUI | agent-device 0.21.14 | 5969.847 | 1.24× | 0% / 0% | github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact |
mobile-agent's benchmark workflow on a GitHub Actions runner: Apple M1 (Virtual), 3 cores, macOS 27.0.1, Xcode 27.1, the iPhone 17 Simulator on iOS 27.0; commit ffe48e1. 10 measured iterations after 2 warm-up, 1 retry; wall clock around each tool's process, nearest-rank p50. These are shared CI runners, no noise study yet, and the host's load average was 519 on its 3 cores when the run started. Not timed: the harness times a tool only once it passes the flow's golden check. github.com/gesposito/mobile-agent/actions/runs/41181272250, harness-simulator artifact, report.md
Per-step gates
Each gate is judged at p95 over the run's measured iterations, and “under” is strict. Values are p95; where a platform has no gate the value is reported beside “no gate”.
| Gate | iOS SimulatoriPhone 17, iOS 26.5, commit ec567d8 | Android emulatorAPI 36 arm64, controller on, -gpu host, commit 60e528a | Physical AndroidNexus 5X, Android 8.1, commit 9769aa0 | Physical iPhoneiPhone SE (3rd gen), iOS 26.6.2, commit 9652e5e |
|---|---|---|---|---|
| CLI process start | 5.88 ms under 20 ms · pass README.md | 11.6 ms under 20 ms · pass README.md | 6.33 ms under 20 ms · pass README.md | SwiftUI phase 6.96 ms Expo phase 7.56 ms under 20 ms · pass README.md |
| Cold attach | SwiftUI 947 ms Expo 838 ms under 2 s · pass README.md | Expo 894 ms Compose 1.21 s under 1.5 s · pass README.md | Expo 13.9 s Compose 3.13 s under 1.5 s · fail app_process alone takes 1.1 to 1.4 s to start on this phone, and two of Expo's 20 opens did not settle within 10 s. README.md | SwiftUI 4.05 s Expo 2.30 s under 5 s · pass README.md |
| Warm snapshot | SwiftUI 7.88 ms Expo 7.65 ms under 150 ms (SwiftUI), 250 ms (Expo) · pass README.md | Expo 12.8 ms Compose 19.4 ms under 100 ms · pass README.md | Expo 7.00 ms Compose 8.69 ms under 100 ms · pass README.md | SwiftUI 8.64 ms Expo 9.38 ms under 1 s · pass README.md |
| Tap dispatch | SwiftUI 1.56 ms Expo 0.178 ms under 20 ms · pass README.md | Expo 2.81 ms Compose 3.32 ms no emulator gate · no gate README.md | Expo 16.4 ms Compose 11.4 ms under 50 ms · pass README.md | SwiftUI 55.1 ms Expo 54.6 ms under 150 ms · pass README.md |
| Tap to frame change | SwiftUI 75.3 ms Expo 52.7 ms no gate · no gate reports/10.2-simulator/report.md | Expo 52.5 ms Compose 55.1 ms under 67 ms, host GPU · pass The run's own report judged these against the 50 ms gate of the time and failed them; task 10.13 restated the gate for host rendering. README.md | no frame stream no device gate · not measured README.md | SwiftUI 364 ms Expo 180 ms no gate while frames are polled screenshots · reported README.md |
Where a command's time goes
Spike S15 profiled the daemon on the Simulator and the emulator. This code's CPU is under 3% of every command that reads or acts. The rest of a click or a fill is waiting: on the device, on the accessibility bridge's tree reads, for frames, and in the quiet windows the replies promise.
| Target | Command | Mean wall | Worker CPU | Share |
|---|---|---|---|---|
| Simulator | click #tab-big, back to back | 645.3 ms | 6.9 ms | 1.1% |
| Simulator | fill alice<n> | 493.0 ms | 3.6 ms | 0.7% |
| Simulator | click text=General | 257.4 ms | 1.4 ms | 0.5% |
| Simulator | wait --settle in Settings | 1555.6 ms | 4.3 ms | 0.3% |
| Emulator | click #tab-big, selected | 214.9 ms | 2.4 ms | 1.1% |
| Emulator | open com.android.settings, cold | 1756.4 ms | 5.6 ms | 0.3% |
Mean ms per request at the daemon's worker thread, from the waiting-thread profile; the share is the worker's CPU over the request's wall time. The bars run to 3%. docs/spikes/s15-profiles.md
This flow, step by step
Open mobileagent.dev in Safari, on an iPhone 17 Simulator, iOS 26.5, with mobile-agent 0.1.0 at ffe48e1. Wall time is each CLI process from start to exit, as the recording script measured it; 9.68 s from the first command's start to the last one's end. Source: src/content/flows/safari.json in this site's repository.
| Step | Command | Wall | Reply (first line) |
|---|---|---|---|
| 1 | open com.apple.mobilesafari | 2.25 s | ✓ Opened com.apple.mobilesafari on udid redacted |
| 2 | click '#TabBarItemTitle' | 0.55 s | ✓ Done (hit text "Search or enter website" (no ref), dispatch 0.0 ms, first frame change 457.2 ms, first tree delta 541.4 ms) |
| 3 | type mobileagent.dev | 1.03 s | ✓ Done (read back "mobileagent.dev", typed) |
| 4 | press Enter | 0.01 s | ✓ Done |
| 5 | wait --settle | 1.44 s | ✓ Settled after 1436.71 ms: the tree changed last |
| 6 | scroll down 2400 | 4.20 s | ✓ Done (4 strokes, at rest 2747.8 ms after the last) |
| 7 | wait --settle | 0.20 s | ✓ Settled after 194.06 ms: the tree changed last |