Table of Contents

Understanding A/B Test Analytics

Samantha Brown Updated by Samantha Brown

A/B test analytics show how each variant of your test performs, from the first widget view to referral revenue, so you can pick a winner with confidence.

What's new

A/B test analytics now track the friend side of your test, so you can test the friend experience on its own:

  • Friend widget views and Friend email submissions show how many friends saw the friend widget and submitted their email.
  • Email submission rate shows the share of friend widget views that led to an email submission.
  • The old Widget views column is now Advocate widget views, so advocate and friend views are counted separately.
  • Your CSV export includes all of the new columns.

All other metrics are calculated the same way as before.

Viewing your results

You need an A/B test that is running or completed. To set one up, see How to Run an A/B Test.

  1. Go to Analytics > Dashboard > A/B Tests.
  2. Select your test and click View Analytics.
  3. Set the date range for the results you want to see.
  4. Choose a tab: Summary, Channels, Placements or Email Metrics. Each tab is described below.
  5. Use Graph Preview below the table to chart a metric over time, with one line per variant.
  6. Click Export to download the current tab as a CSV file. The file includes the same columns as the table.
  7. Click Preview Variants to see what customers see in each variant.

Reading the comparison row

Under the variant rows, a comparison row (for example, B Over A) shows how much higher or lower each variant is than Variant A:

B over A = (Variant B − Variant A) ÷ Variant A

For example, if Variant A has 200 shares and Variant B has 250, B over A is +25%.

Summary and Placements metrics

The Summary tab shows totals for each variant. The Placements tab shows the same metrics broken out by widget placement.

Metric

What it shows

How it's calculated

Advocate widget views

Times the advocate (sharing) widget was viewed

Count

Shares

Shares made by advocates, including email shares. Reminder emails aren't counted.

Count

Share rate

How often an advocate widget view led to a share

Shares ÷ Advocate widget views

Clicks

Clicks on referral links. Every valid click is counted, not just unique clicks.

Count

Friend widget views

Times the friend widget was viewed

Count

Friend email submissions

Emails friends submitted through a referral. Each email address counts once.

Count

Email submission rate

How often a friend widget view led to an email submission

Friend email submissions ÷ Friend widget views

Sign ups

Referred friends who signed up

Count

Sign up rate

How often a click led to a sign up

Sign ups ÷ Clicks

Purchases

Purchases by referred friends. Test orders and recurring orders aren't counted.

Count

Conversion rate

How often a click led to a purchase

Purchases ÷ Clicks

Referral revenue

Total value of purchases by referred friends. Test orders aren't counted.

Sum of purchase amounts

Email submission rate uses friend widget views, not total widget views, so it reflects only the friends who saw the friend experience. For example, 120 friend email submissions from 1,000 friend widget views is a 12% email submission rate.

Email submission rate appears in the tables only. You can't select it in Graph Preview.

Channels metrics

The Channels tab breaks results out by share channel, such as email, SMS, social share or a PURL. Widget views and email submissions aren't shown here because they happen before an advocate picks a channel.

Metric

How it's calculated

Shares

Count

Clicks

Count

Sign ups

Count

Sign up rate

Sign ups ÷ Clicks

Purchases

Count

Conversion rate

Purchases ÷ Clicks

Referral revenue

Sum of purchase amounts

These metrics are defined the same way as on the Summary tab.

Email Metrics

The Email Metrics tab shows how the referral emails in each variant perform. Share emails are sent by advocates to friends; reminder emails are follow-ups sent after the original share.

Metric

What it shows

How it's calculated

Shares

Share emails sent by advocates (share emails only)

Count

Recipients

Friends the share emails were sent to (share emails only)

Count

Emails sent

Reminder emails sent (reminder emails only)

Count

Delivery rate

How many emails were delivered

Delivered ÷ Recipients (share) or Delivered ÷ Emails sent (reminder)

Unique opens

Emails opened at least once

Count

Open rate

How many delivered emails were opened

Unique opens ÷ Delivered

Total clicks

All clicks on links in the emails

Count

Unique clicks

Emails clicked at least once

Count

Click thru rate

How often a delivered email was clicked

Total clicks ÷ Delivered

Sign ups

Friends who signed up from an email

Count

Sign up rate

How often an email click led to a sign up

Sign ups ÷ Total clicks

Purchases

Friends who purchased from an email

Count

Conversion rate

How often an email click led to a purchase

Purchases ÷ Total clicks

Referral revenue

Total value of those purchases

Sum of purchase amounts

On this tab, Sign up rate and Conversion rate use email clicks, so they can differ from the same metrics on the Summary tab.

FAQ

Why do I see a dash (-) instead of 0%? A dash means there's nothing to calculate from. For example, a variant with zero clicks shows a dash for Sign up rate. A 0% means the activity happened but didn't lead to a result, such as friends viewing the widget without submitting an email.

A friend submitted the same email twice. Why didn't the count go up? Each email address counts once. This is the same rule Friendbuy uses for Sign ups.

Why don't the Summary and Email Metrics tabs show the same Sign up rate? The Summary tab divides by all referral link clicks. The Email Metrics tab divides by clicks in the emails only.

Why are some referral customers missing from my test results? Referral links created before the test started keep showing the original offer, so those referrals aren't part of the test. See How to Run an A/B Test.

Do the new columns appear in my CSV export? Yes. The export matches the columns shown in the tables.

Still have questions? Contact your Friendbuy Customer Success Manager or email support at support@friendbuy.com.

How did we do?

How to Archive a Referral Campaign

How to Duplicate Your Campaign

Contact