Skip to content
Sandworld
Book a pilot

Documentation

Build with Sandworld

Reset, step and submit from any agent: REST, MCP, Harbor packages and a local server.

Android emulator

Phone agents are tested on emulators, but a real app on an emulator can't be reset to an exact state, has a live backend that takes real orders and payments, and gives no way to check what actually happened. The Sandworld Twins app fixes that: it runs any mobile twin full screen on an Android emulator or device, like an installed app, while a real Sandworld episode records every tap. When the agent is done, the task's verifier checks the final state, the same checks an API or browser agent gets.

It works with anything that drives Android: adb, UI Automator, AndroidWorld-style harnesses, or a model that reads the screen.

1. Install the app

# Download the APK (debug-signed, for emulators and test devices)
curl -LO https://sandworld.dev/files/sandworld-twins.apk
adb install -r sandworld-twins.apk

# or build it from the repository with the Android SDK (no Gradle needed)
sh android/build.sh

2. Open a task

adb shell am start -S -n dev.sandworld.twins/.MainActivity \
  --es world quickbasket --es task qb-01 --ei seed 7 \
  --es server https://sandworld.dev        # or http://10.0.2.2:3000 for a local server

The app starts a fresh episode and writes it to logcat under the tag Sandworld:

{"event":"episode","episode_id":"ep_…","env":"quickbasket","task_id":"qb-01","seed":7,
 "instruction":"Order 2 cartons of Whole Milk (1 L) and one 12-pack of Free-Range Eggs…"}
{"event":"step","step":1,"tool":"search_products","ok":true}

3. Observe and act

The screen is read through Android's accessibility tree (uiautomator dump): every control comes with its role, its accessible name (“Add Whole Milk”, “Proceed to checkout”) and its bounds. Actions are ordinary adb input: taps, text, swipes and key presses. The first dump after launch switches on the WebView's accessibility tree, so ask twice.

4. Verify

curl -s -X POST https://sandworld.dev/api/play/episodes/$EPISODE/submit

The answer is the task's verifier result: pass or fail, a score, and every check with its detail.

The Python harness

sandworld.android in the Python client wraps all of this. Standard library only; it needs adb and a running emulator.

from sandworld.android import AndroidTwin

twin = AndroidTwin("quickbasket", task_id="qb-01", seed=1, server="https://sandworld.dev")
obs = twin.reset()                      # opens the app and starts the episode
print(obs["instruction"])
for el in obs["elements"]:              # role, label, bounds and centre of every control
    print(el["index"], el["role"], el["label"])

twin.tap_text("Search for milk")        # or twin.tap(el), twin.tap((x, y))
twin.type_text("milk", submit=True)
twin.tap_text("^Add Whole Milk$")
twin.swipe("up")

result = twin.submit()                  # the task's verifier
print(result["pass"], result["score"])

python -m sandworld.android selftest --server https://sandworld.dev places the Quickbasket order from task qb-01 entirely through the phone screen and prints the verifier's checks; all seven pass.

What runs on Android

  • The mobile twins: Quickbasket (grocery delivery), Coinpurse (payments wallet) and Skyfare (airline), with every task and seed.
  • The same verifiers, step budgets and policies as the API, so tool-use, browser and phone results on one task compare directly.
  • Any Android 8.0+ emulator or device. The app is a WebView shell around the twin, so a twin change ships to phones without a new APK.

For browser-based computer use (screenshots, Set-of-Marks, clicks in a desktop browser), see Computer use.