Automation · Operations and reporting
Review an A/B experiment's result in Amplitude on WhatsApp
Ask on WhatsApp how an experiment is doing: an AI agent reads each variant, its significance and the funnel in Amplitude.
How do I know whether a variant of my experiment has won without opening Amplitude?
You write the experiment's name to your Met, for example "annual price with discount". The Met finds it in Amplitude, brings its state and variants, queries the analysis with each variant's performance and its statistical significance, and checks the purchase funnel in the same period to see whether the winning variant also moves the final conversion. It tells you whether there is a winner or more traffic is needed, and leaves a comment in Amplitude with the summary if that action is on. It does not change or stop the experiment.
What starts it
When someone asks for it
It starts when someone on the team asks the Met about an experiment. Each review runs on request; this procedure does not leave a scheduled alert.
A manual task: someone on the team runs it with a button, or asks the Met for it in chat.
Who runs it
Product analytics Met with Amplitude
This is not a named, seeded Met. It is an internal Met with Amplitude connected: its work ends when it replies with what the experiment analysis says, or when it says there is not enough data to decide yet.
Channels: WhatsApp
What you need connected
- Amplitude Research and analyze
5 steps · 5 tools
The procedure, step by step
Each step shows the tool that runs. They belong to the real connector: if one of them stopped existing, this page would not build.
- 1
Find the experiment
The Met searches for the experiment by name in Amplitude. If several similar ones show up, it asks you which one instead of picking the first.
- 2
Bring its state and variants
The Met brings the experiment with its state, each variant's configuration and the decision, if there is one. That tells it since when it runs and which metric it is measured against.
- 3
Read each variant's analysis
The Met queries the experiment analysis: how each variant does on the primary metric and whether the difference is statistically significant according to Amplitude.
- 4
Check the funnel in the same period
So it does not decide only on the experiment metric, the Met runs a purchase funnel query for the same period and tells you whether the final conversion moved.
- 5
Reply and leave the summary in Amplitude
The Met replies on WhatsApp whether there is a winner or more traffic is needed. If the comments action is on, it leaves the same summary as a comment in Amplitude for the rest of the team.
Before you start
What you need ready
The Met solves none of these for you. If one is missing, the automation stops there.
-
An experiment running in Amplitude
An experiment created in Amplitude with its primary metric and traffic coming in. Without data, the analysis has nothing to say.
-
Amplitude connected in Met
In Met, open Connections → Amplitude → Connect and sign in with an account that sees the project. For the comment, the admin leaves that action on.
-
An internal Met your team writes to
A Met with Amplitude enabled and a WhatsApp line your team uses to write to it. It is an internal query, not a customer conversation.
The limits
What this automation does NOT solve
Better said here than discovered halfway through the rollout.
-
It does not stop or change the experiment. Choosing the winner and turning off the losing variant is done in Amplitude, by a person on the team.
-
It does not declare a winner without significance. If Amplitude's analysis says the difference is not significant, the Met says so as it is.
-
It does not ask for confirmation in the chat before leaving the comment: if the action is on, it leaves it. The admin decides which actions stay on.
-
It does not work with accounts in the Europe region. The connector uses Amplitude's standard server.
FAQ
Questions about this automation
Yes, one after another. Naming them well in Amplitude helps the search find the right one on the first try.
The analysis usually comes out without significance. The Met tells you and suggests asking again when there is more traffic.
It shows the Amplitude account that authorized the connection in Met, not the person who wrote to the Met on WhatsApp.
Keep going
Other automations built on the same tools
Operations and reporting
Turn off on WhatsApp the feature flag that breaks the checkout in LaunchDarkly
How do I quickly turn off a feature flag that broke the checkout without opening LaunchDarkly?
- Starts:
- when someone asks for it
- Steps:
- 5, with 7 real tools
- Needs:
- LaunchDarkly
Operations and reporting
Build a department's higher education report without opening SNIES
How do I pull SNIES enrollment and program figures without going into the portal?
- Starts:
- when the customer writes
- Steps:
- 5, with 5 real tools
- Needs:
- SNIES
Operations and reporting
Build your team's org chart in Lucid on WhatsApp and share it
How do I build my business's org chart without sitting down to draw it?
- Starts:
- when someone asks for it
- Steps:
- 6, with 6 real tools
- Needs:
- Lucid
Operations and reporting
Tell customers on your status page that the store is down, asking on WhatsApp with Better Stack
How do I tell my customers my online store is down without opening Better Stack?
- Starts:
- when someone asks for it
- Steps:
- 6, with 6 real tools
- Needs:
- Better Stack
Want this automation running on your own data?
We will show it to you with your accounts connected, not with a canned demo.