Amplifier bundle · now public

ios-tester

The iOS counterpart to android-tester — boot a simulator, launch an app, drive its UI, and verify what actually rendered.

github.com/microsoft/amplifier-bundle-ios-tester

What it is

One tool. Three agents. Two backends.

ios_inspector · 24 operations

Simulator lifecycle (create_sim, boot, install, launch) · sensing (screenshot, ui_dump, find, logs) · interaction (tap, type_text, swipe, wait_for) · a read-only physical-device tier

Specialist agents

ios-operator — drives the app
ios-visual-tester — visual & layout
ios-debugger — root-cause

The rule everything rests on

The accessibility tree is the sensor.
The screenshot is for judgment — never targeting.

Ask a vision model where an element is and it answers confidently. It is also wrong in a way iOS will never report back to you.

tap resolves a selector against a fresh tree before acting. Raw-coordinate tapping exists as a separate tap_xy that always warns.

Measured · Settings, the “General” row

130px off. Rows are 156px apart.

Apple Account

General — the intended target

Accessibility

130 px

Source	Center y
accessibility tree	1210
vision model	1080
delta	−130 px
row spacing	156 px

The estimate lands cleanly on the row ABOVE.

iOS raises no error. The tap succeeds, the wrong thing happens, and the screenshot still looks entirely plausible.

The iOS-only trap — handled for you

The tree speaks points. Screenshots speak pixels.

accessibility tree

393 × 852

points

× 3.0

screenshot

1179 × 2556

pixels

Mixing them is a silent 3× coordinate error. Every geometric field comes back in both spaces, explicitly labelled — frame_points / frame_pixels, center_points / center_pixels — plus the measured scale. The tool never returns a bare coordinate whose unit is ambiguous.

What a resolved tap looks like

Evidence, not assertion.

node      "General"   frame_points  [16.0, 380.33, 370.0, 52.0]
tap       →           points        (201.0, 406.33)
                      pixels        (603.0, 1219.0)

201.0 × 3.0 = 603.0        406.33 × 3.0 = 1219.0

The tap coordinate is exactly the computed centre of the resolved node. It came from the tree — and the result proves it did.

Three tiers · honestly labelled

What each tier can actually do.

Tier	Setup cost	Can see	Can tap
Simulator	Xcode only	yes — labels + frames	yes · proven
Device — free	cable + Developer Mode	yes — screenshot, elements, syslog	no
Device + WebDriverAgent	+ Apple Developer account, signing	yes	deferred · unproven

The simulator tier needs no Apple Developer account, no code signing, and no physical device. That is the tier nearly everyone will run.

Why the device tier refuses to tap

No geometry. So it will not guess.

The free-tier device element list returns exactly four keys:

caption · estimated_uid · platform_identifier · spoken_description

No frame. No rect. No bounds. Nothing positional at all.

Tapping from captions alone is precisely the silent-miss failure on the earlier slide. So tap on the device backend returns a structured refusal explaining why — rather than guessing a coordinate and reporting success.

Where it runs

On the Mac, or driving one remotely.

Runs natively on macOS
Drives a remote Mac over plain SSH — so a Linux- or WSL-hosted agent reaches iOS the same way
Headless simulator boot measured at 2.7 seconds over bare SSH

If xcode-select points at CommandLineTools, simctl appears missing. Setting DEVELOPER_DIR fixes it — no sudo required. doctor detects this and tells you.

Limits, stated plainly

What it will not pretend to do.

Physical devices cannot be tapped on the free tier — no geometry exists, so the tool refuses rather than guessing.
type_text cannot confirm focus was gained. iOS exposes no focused attribute. It asserts readback and warns, rather than implying certainty.
WebDriverAgent tapping is deferred and unproven. It would cost an Apple Developer account and code signing.

Everything above is a known boundary, surfaced by the tool at the moment you hit it — not a surprise discovered halfway through a test run.

Verification

Numbers, not adjectives.

165

unit tests

18

files · 0 errors

PASS

bundle validation

PASS

MS repo audit

Proven end-to-end against macOS 26.6 / Xcode 26.6 and a physical iPhone X on iOS 16.7.16 — not only unit-tested.

Get it

Install, then run `doctor`.

amplifier bundle add \
  git+https://github.com/microsoft/amplifier-bundle-ios-tester@main#subdirectory=behaviors/ios-tester.yaml \
  --app

doctor reports every host problem at once — Xcode, simctl, runtimes, simulators, devices, Developer Mode — each with its own fix.