AdAce AdsSet advertising test stop rules before the first observation
A test can end because it answered its question, because its authorized observation budget ran out or because continuing would create an unacceptable operational risk. These are different outcomes. Prewriting stop rules makes the distinction explicit and reduces the temptation to label any convenient endpoint success. The rules should be understandable to the person monitoring the test without requiring them to invent policy during an incident.
Stopping has more than one meaning
Create four rule categories: planned evaluation, exposure limit, measurement invalidity and operational emergency. Planned evaluation defines when mature results will be assessed and under what decision criterion. Exposure limits control approved cost or duration. Measurement rules identify circumstances that invalidate comparisons, such as a broken outcome event or unequal data coverage. Emergency rules protect operations, such as an unavailable destination or sales-team inability to handle enquiries.
Educational example, not a budget recommendation: an authorized illustrative test has a total observation limit of 1,200 units across both variants and a scheduled review after its chosen maturation window. By the checkpoint, spend is 1,180 and evidence remains inconclusive. The operator does not authorize another 1,200 just because a winner has not appeared. The record says stopped near exposure limit, hypothesis unresolved. A separately authorized extension would be a new decision with documented rationale.
Make each rule measurable: condition, data source, person responsible, action and notification. For example, an outcome-event outage confirmed in both the event log and a controlled internal check can invalidate a comparison. A poor provisional CPA is different: if outcomes are immature, it may not satisfy any performance-stop criterion. Do not interchange delivery monitoring with statistical assessment. Design formal interim decisions with an appropriate method rather than repeatedly applying an ordinary final-test rule.
Separate learning from protection
- A neutral unresolved status protects learning quality. It prevents a forced winner from entering the next campaign as if the evidence were conclusive.
- An exposure limit makes the cost of information visible. A business may reasonably choose not to buy enough information to settle every possible hypothesis.
- Separate emergency ownership from interpretation ownership. An operator can protect users and spending while the analyst later decides whether any surviving data remains interpretable.
Write rules the operator can apply
- Define the planned endpoint and outcome maturity rule before observation begins. State what would justify adopting, rejecting or leaving the treatment undecided.
- Record the authorized aggregate exposure and how actual spend will be monitored. For Google, average daily budget settings are not a guaranteed hard daily spending ceiling; base the rule on the relevant budget model and observed exposure.
- List data-validity conditions, including outcome definition, event continuity, coverage balance and treatment consistency. Specify whether invalid data causes a temporary pause or ends the comparison.
- Write emergency conditions and the person authorized to act. Record the operational action separately from any claim about relative campaign performance.
- After stopping, save the reason, time, spend, mature outcome counts and unresolved assumptions. Use AdAce Ads records for read-only documentation; a stop instruction or other tool write requires its own authorized workflow.
An early stop may leave the hypothesis unanswered
- A spend cap does not establish statistical power. A test can be affordable yet too small to answer the selected question.
- Data recovered after an outage may not reconstruct missing exposure or restore comparable groups. Decide explicitly whether the original analysis remains valid.
- If you change a rule after observing results, retain the original and the amendment. Explain the operational reason rather than presenting a revised endpoint as prespecified.
Sources and further reading
Try it on your own accounts
Create a workspace, connect Google or Meta in a couple of clicks and see your accounts clearly. Changes follow your approvals or the policy you configure.
Create your workspace