Automation · Operations and reporting
Find out on WhatsApp which Optimizely experiments won this quarter and what to repeat
An AI agent tells you on WhatsApp which Optimizely experiments won this quarter, with lift and significance, and how the active one is doing.
How do I know which Optimizely A/B tests won this quarter without building a report?
You ask your Met on WhatsApp which experiments won this quarter. The Met pulls from Optimizely the top performing experiments among paused and concluded ones, and runs a query on their results to give you the lift, significance and visitors for the metric you care about. Then it checks which experiments are still running and sums up how each one is doing. With that it tells you which change clearly won, which did not reach significance and which is worth repeating with another variation. It changes nothing in Optimizely: it only queries.
What starts it
When someone asks for it
It starts when the owner, the product team or the agency asks the Met on WhatsApp how the quarter's experiments did. There is no trigger to configure: the request arrives in the conversation.
A manual task: someone on the team runs it with a button, or asks the Met for it in chat.
Who runs it
Experiments analyst Met
This is not a named, seeded Met. It is the internal Met of the business or its agency, with Optimizely connected. Its work ends when you know which experiments won, with which numbers, and which one is worth repeating.
Channels: WhatsApp
What you need connected
- Optimizely Research and analyze
5 steps · 5 tools
The procedure, step by step
Each step shows the tool that runs. They belong to the real connector: if one of them stopped existing, this page would not build.
- 1
Pull the top performing experiments
The Met asks Optimizely for the program's top performing experiments. That report covers paused and concluded ones, which are the ones that already have a result to compare.
- 2
Get the numbers for the metric that matters
The Met runs a query on those experiments' results for the metric you named, such as purchases or sign ups, and returns lift, significance, confidence intervals and visitors.
- 3
Understand how the data is structured
Before querying what is still running, the Met gets the schema of the experimentation entities, to learn what projects, flags and experiments are called in your account.
- 4
List what is still running
The Met runs a query on your experimentation data to list the project's active experiments, which the top experiments report does not include.
- 5
Sum up how each active experiment is doing
For each active experiment, the Met returns and analyzes its result, and tells you in the chat whether a variation is already ahead or there is not enough data to decide yet.
Before you start
What you need ready
The Met solves none of these for you. If one is missing, the automation stops there.
-
Optimizely with Agent Platform
An Opti ID and an Optimizely account with Agent Platform enabled and linked to a Feature or Web Experimentation instance, with concluded experiments to compare.
-
Optimizely connected in Met
In Met, open Connections → Optimizely → Connect, sign in with your Opti ID and pick the Agent Platform instance. This procedure only uses query tools.
-
The metric and the project in the skill
Write in the Met's skill which Optimizely project and main metric to use, so it compares experiments by what really matters to the business.
The limits
What this automation does NOT solve
Better said here than discovered halfway through the rollout.
-
It does not launch or stop experiments. This procedure only queries; creating or changing an experiment starts switched off and is a separate decision for the admin.
-
It does not decide for you which change to ship. It gives you Optimizely's numbers and which one won; shipping the winning variation on your site is still your team's job.
-
It does not make up significance. If an experiment does not have enough visitors, the Met tells you so with Optimizely's data instead of declaring a winner.
-
It does not see experiments from other accounts or instances that the connected account does not have linked in Agent Platform.
FAQ
Questions about this automation
Because that Optimizely tool works with paused and concluded experiments. That is why this procedure queries the active ones separately and sums up each one.
Yes. Name it in the chat and the Met runs the query on that metric, as long as it is defined in the Optimizely experiments.
Yes, if the connected account has access to their projects. The Met queries the project you name in the chat or the one written in the skill.
Keep going
Other automations built on the same tools
Operations and reporting
Turn off on WhatsApp the feature flag that breaks the checkout in LaunchDarkly
How do I quickly turn off a feature flag that broke the checkout without opening LaunchDarkly?
- Starts:
- when someone asks for it
- Steps:
- 5, with 7 real tools
- Needs:
- LaunchDarkly
Operations and reporting
Build a department's higher education report without opening SNIES
How do I pull SNIES enrollment and program figures without going into the portal?
- Starts:
- when the customer writes
- Steps:
- 5, with 5 real tools
- Needs:
- SNIES
Operations and reporting
Build your team's org chart in Lucid on WhatsApp and share it
How do I build my business's org chart without sitting down to draw it?
- Starts:
- when someone asks for it
- Steps:
- 6, with 6 real tools
- Needs:
- Lucid
Operations and reporting
Tell customers on your status page that the store is down, asking on WhatsApp with Better Stack
How do I tell my customers my online store is down without opening Better Stack?
- Starts:
- when someone asks for it
- Steps:
- 6, with 6 real tools
- Needs:
- Better Stack
Want this automation running on your own data?
We will show it to you with your accounts connected, not with a canned demo.