Argent Device Interact
Interact with an iOS simulator, Android emulator, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. Not for TV targets.
- Skill ID
- software-mansion/argent/argent-device-interact
- Publisher
- software-mansion
- Repository
- argent
- Installs
- 287
- Files
- 4
- Synced
- Sep 16, 2026
Open any RiverX project, open the Skills panel in the chat, and search for this identifier. The files are fetched from the source repository at install time.
software-mansion/argent/argent-device-interactInstalls these files- SKILL.md
- references/chromium.md
- references/gesture-examples.md
- references/secrets.md
What this skill tells the agent
Unified tool surface
All interaction tools below accept a udid parameter and auto-dispatch iOS vs Android based on its shape (UUID → iOS simulator, chromium-cdp-<port> → Chromium (CDP) app, anything else → Android adb serial). You use the same tool names on every platform.
Chromium (CDP) app = an Electron app or a Chromium-family browser (Chrome/Brave/Edge) exposing a Chrome DevTools Protocol endpoint. The same describe/tap/keyboard/screenshot surface drives it, but scrolling, tabs, cookies and storage differ — read `references/chromium.md` before driving a `chromium` target.
1. Before You Start
If you delegate simulator tasks to sub-agents, make sure they have MCP permissions.
Use list-devices to get a target id. Results are tagged with platform (ios, android, or chromium); booted/ready devices come first. Pick the first entry that matches the platform you need — if none are ready, call boot-device with udid (iOS), avdName (Android), or electronAppPath (boots an Electron app as a chromium device). A Chromium browser already running with a CDP port shows up directly — no boot-device needed. See argent-ios-simulator-setup / argent-android-emulator-setup for full setup flow.
Load tool schemas before first use. Gesture tools (gesture-tap, gesture-swipe, gesture-pinch, gesture-rotate, gesture-custom) may be deferred — their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use before calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors.
2. Best Practices
- Always refer to tapping_rule from your argent.md rule before tapping.
- Before performing interactions, consider whether they can be dispatched sequentially - more on that in
run-sequence. - Use `gesture-swipe` for lists/scrolling, not
gesture-custom, unless you need non-linear movement. On Chromium usegesture-scrollinstead —gesture-swipeis touch-only. Consider whether you need multiple swipes, if yes - userun-sequence. Passmomentum: falsewhen the swipe should decelerate before ending for a precise movement. - Tap a text field before typing, then use
keyboardto enter text. - Coordinates are normalized — always 0.0–1.0, not pixels.
- For app navigation, use the element tree returned after each action (
--- Elements after action (describe) ---); calldescribeonly when no fresh tree is available for the current screen. It works on any screen without app restart. Do not navigate from screenshot pixels on regular in-app screens unless the tree failed to expose a reliable target. Usenative-describe-screenonly when you need app-scoped UIKit properties.
3. Opening Apps
Never navigate to an app by tapping home-screen icons. Use launch-app or open-url — they are instant and reliable.
launch-app — by bundle ID
{ "udid": "<UDID>", "bundleId": "com.apple.MobileSMS" }Common IDs: com.apple.MobileSMS (Messages), com.apple.mobilesafari (Safari), com.apple.Preferences (Settings), com.apple.Maps, com.apple.Photos, com.apple.mobilemail, com.apple.mobilenotes, com.apple.MobileAddressBook (Contacts)
open-url — by URL scheme
{ "udid": "<UDID>", "url": "messages://" }Common schemes: messages://, settings://, maps://?q=<query>, tel://<number>, mailto:<address>, https://... (Safari)
4. Choosing the Right Tool
| Action | Tool | Notes |
|---|---|---|
| Multiple actions | run-sequence | Batch steps in one call (no intermediate screenshots) |
| Open an app | launch-app | Always — never tap home-screen icons |
| Restart an app | restart-app | Terminate and relaunch by bundle ID |
| Open URL/scheme | open-url | Web pages, deep links, URL schemes |
| Single tap | gesture-tap | Buttons, links, checkboxes |
| Scroll/swipe | gesture-swipe | Straight-line scroll or swipe |
| Scroll (Chromium) | gesture-scroll | Wheel-based; deltas are window fractions, positive deltaY = down |
| Drag (Chromium) | gesture-drag | Sliders, drag-and-drop, text selection |
| Long press | gesture-custom | Context menus, drag start |
| Drag & drop | gesture-custom | Complex drag interactions |
| Pinch/zoom | gesture-pinch | Two-finger pinch with auto-interpolation |
| Rotation | gesture-rotate | Two-finger rotation with auto-interpolation |
| Custom gesture | gesture-custom | Arbitrary touch sequences, optional interpolation |
| Hardware key | button | Home, back, power, volume, appSwitch, actionButton |
| Type text | keyboard | Every platform. Text or one named key per call, never both |
| Paste text | paste | Only where a user would paste (OTP code, long link). Sim/emu only |
| Rotate device | rotate | Orientation changes |
| Shake device | shake | Shake handlers (sim/emu only), Undo-typing prompt, RN dev menu |
| Wait for UI | await-ui-element | Block until an element is visible/hidden/exists/contains text |
| Wait for idle | await-screen-idle | Block until a non-empty screen tree stops changing |
5. Finding Tap Targets
IMPORTANT. When moved to a different screen after an action or do not know the coordinates of component, always perform proper discovery first.
| App type | Discovery tool | What it returns |
|---|---|---|
| Target app discovery | describe | Accessibility element tree for the current device screen (iOS AX-service, Android uiautomator, or Chromium DOM walker) with normalized frame coordinates. Works on any app, system dialogs, and Home screen — no app restart or bundleId required |
| React Native | debugger-component-tree | React component tree with names, text, testID, and (tap: x,y) |
| App-scoped native | native-describe-screen | Low-level app-scoped accessibility elements with normalized and raw coordinates; requires bundleId |
