Automation
Make your taps survive tomorrow
The four properties a flow can recognise a button by, the four ways to compare them, AND/OR groups — and why coordinates are always the last resort.
6 min read
Here is the most common story in automation. You build a flow, it runs beautifully, you go to bed happy. A week later it fails on half your devices and nothing about your flow has changed.
Almost always, what changed is the screen — and what broke is how your flow was recognising the button. That recognition is called a matcher, and choosing it well is the difference between a flow that lasts a month and one that lasts a day.
What a matcher actually is
Every button, input box and line of text on the phone is an element. A matcher describes one of them well enough that the device can find it again on its own.
The Click node puts the choice in front of you in one sentence: "Either tap a fixed coordinate, or let the device find the element and tap it."
A coordinate is a promise about pixels. A matcher is a description of a thing. Screens change size; things keep their names.
The fastest way to build one is not to type it at all — the UI Inspector writes it from the element you click on.
The four things you can describe
| Prop | What the app says about it |
|---|---|
| resource-id | "An identifier set by the app. The most stable one; use it whenever it exists." |
| text | The visible text. Easy to read but changes with language and content. Defaults to a case-insensitive "contains" match. |
| content-desc | "The accessibility description, usually present on icons without text. Matched like text." |
| class | "The widget type (Button, EditText…). Many elements share it, so only use it to narrow down further." |
resource-id is a name the app's own developers gave the button. Users never see it, and that is exactly why it survives: nobody rewrites it for a marketing campaign, and it does not get translated.
text is the opposite — the easiest to read and the easiest to break. Point a flow at the words Sign in and it stops working the day the phone is set to Vietnamese and the button says Đăng nhập.
content-desc is the label a screen reader announces. Icons with no writing on them — a heart, a paper plane, three dots — usually have one, and often it is the only handle they offer. A hint elsewhere in the editor makes the same point from the other side: "Many apps only set the content-description, not the text — pick “Description” when text comes back empty."
class on its own is nearly useless: one screen can hold dozens of Buttons. As a second condition it is excellent.
The Inspector's tips suggest reaching for text or content-desc first, and falling back to resource-id or class when those are missing. That is advice about what you can see: those two are the props you read straight off the screenshot and verify with your own eyes.
The prop descriptions answer a different question — what is least likely to change — and there the answer is resource-id.
Both are useful. Build the matcher from what you can see, then ask one question before moving on: would this still be true on a phone set to another language? If not, and a resource-id exists, switch to it.
The four ways to compare
Having picked a prop, you say how its value should be compared against what you typed.
| Comparison | What it does |
|---|---|
| contains | "The prop value contains the entered string. Tolerates extra text on either side, so prefer it." |
| equals | "The prop value must be exactly the entered string." |
| starts with | "The prop value begins with the entered string. Handy for text with a trailing counter." |
| matches regex | "Regular-expression match. Only when the three above are not enough." |
contains is what text falls back on by default, and the editor's own advice is to prefer it. Apps love to pad their labels — a leading space, a trailing badge, an invisible character — and equals fails on every one of those while contains shrugs them off.
starts with earns its keep on anything that counts. A tab reading Messages (3) today and Messages (11) tomorrow is still the same tab, and Messages as a starts-with catches both.
matches regex is a pattern language — an advanced search expression where symbols stand for whole families of text. The pattern ^Item \d+$ means "the word Item, a space, then any number, and nothing else". It is powerful and easy to get subtly wrong, which is why the app itself says to reach for it only when nothing simpler works.
Any of the four can be modified two ways: Ignore letter case, and negation — "Every operator can be combined with Ignore case and Negate (must NOT match)." Negation is for screens you identify by what is absent: a row without a verified badge, a field with no error message under it.
When several elements match
Sometimes several elements legitimately match and you want a particular one. The Position field handles that, and its hint is exact: "If several elements match, pick which one — counting from 0. Empty means the first one."
Counting from zero means the first is 0 and the second is 1. Useful for "the third result in this list" — but treat it as a late resort, because that order belongs to the app, not to you.
AND, OR, and the language problem
One condition is usually plenty. When it is not, conditions go into groups:
"A group holds several conditions. Inside a group choose AND (all must hold) or OR (one is enough)."
Between groups choose AND or OR as well. Example: Group 1 text contains "Sign in" OR Group 2 description contains "Login" handles both app versions.
That example is worth reading twice, because it cures the problem this page opened with. AND narrows: text contains Next AND class is Button, for a screen that has both a "Next" heading and a "Next" button. OR widens — and widening across two labels is how one flow survives two languages, two app versions, or an A/B test nobody told you about.
The app's closing advice on groups is restraint: "One condition is enough in most cases. Add groups only when the screen has several look-alike elements." Every condition you add is one more thing that can stop being true.
The order to try things in
A resource-id that reads like a name. Nothing a user does changes it.
text or content-desc, with contains. Readable, verifiable in the Inspector, good enough for most screens.
Two of them together, or an OR group across both. For look-alike elements, and for apps that run in more than one language.
Position, or regex. When the screen really does offer nothing else.
A coordinate. Last. Always last.
Coordinates are the one option that cannot survive being moved. The Inspector says it plainly: "Fall back to coordinates only when nothing else works; they break on another device or resolution."
A coordinate bets that the next phone has the same screen size, the same system font size, and no notification bar in the way. Run that flow across a batch of devices and it will tap something — just not always the thing you meant. Keep coordinates for canvases, maps and games, where there is genuinely nothing to name.
Summary
- A matcher describes an element; a coordinate describes a pixel. Prefer the description.
- resource-id changes least, text and content-desc are easiest to verify — build with what you can see, then ask whether it survives another language.
- contains by default, starts with for labels with counters, equals when it must be exact, regex only when nothing simpler works.
- AND narrows, OR widens — an OR group across two labels is how one flow handles two languages.
- Coordinates last: they break on another device or resolution.