How to Test an Automation Against Past Activity

Before an automation goes live, replay it over stored history to see how often it would have run. Nothing is sent or changed: it is a replay of stored history.

Before you start

You need the Manage IoT Automations right and an automation open in the builder. It does not need to be saved: the replay uses your unsaved changes.

Replay

At the bottom of the builder, choose Replay history.
Set the period with From and To. It starts at the last 30 days. History older than your organization keeps is not replayed.
Choose Replay.

Read the result

It would have run N times on N devices, with a chart of the Times it would have run each day.
The first N times lists up to 20 of them, so you can see whether it ran when you expected.
Each trigger shows how complete the history was: Exact, Partial or No history. Readings between limits are not stored, so a new rule on a reading such as a temperature is often Partial.
Devices you cannot see are left out, and the result says how many.
The replay also applies Shortest time between runs (seconds) and Most runs per hour.

Go further with Watch only

When the history is not exact, choose Turn on in Watch only mode, then save the automation. It then records what it would do in real time without doing it, and the next replay is exact. When you are happy, change How it runs to Live.

Test the steps once

Test runs a saved automation once without changing anything and shows what each step would do. Tick Send me the notification to receive its notification yourself, then choose Run the test. Save your changes first: a test does not use unsaved edits.

If it doesn't work

"The history was too long to replay completely. Pick a shorter period." Choose a shorter period. A replay stops after 15 seconds or 50,000 records.
A trigger that covers more than 200 devices cannot be replayed. Narrow what it applies to.
You can run up to 6 replays a minute.

Good to know

A replay can only be as good as the history kept. See IoT Settings: Retention, Re-check and Defaults.