June 29, 2015

Optimizing Tweets: Mayo Clinic’s Lessons From A/B Testing

By Tony Hart, Champion Specialist

ab-testing.500x239pxIn 2015, Mayo Clinic’s Social and Digital Innovation team ran two months of A/B testing to better understand how our followers respond to @MayoClinic tweets.

A/B testing is a way to compare two versions of an ad or marketing message to see which is more successful in getting your target audience to do a desired action (e.g., clicking the ad, entering an email address). Basically, A/B testing will allow you to tweak future messaging to make marketing more successful. Click here for more information on A/B Testing. Here's what we learned:

General Twitter Guidelines

  • Use strong hashtags, ones that are on topic and frequently used (e.g., #hcsm). Do some quick research about how others currently use that hashtag.
  • Include eye-catching images to get your audience to stop scrolling through their feed to view your image and the content along with it.

Tweeting for Engagement (RTs, Replies, Favs)

  • Post facts, tips, and bold statements. Here is an example that received 311 engagements (almost double any of the other tweets), but only 371 clicks (a little over half of the top post in terms of clicks):

  • Use "tell-all" images to reinforce message and increase engagement. Infographics and medical illustrations work well, as do high quality close-ups, unique angles, and colorful images.

When Tweeting for URL Click-throughs

  • Use teaser text to highlight what readers will learn when clicking on the link. Here is an example that received 626 clicks but only 153 engagements (less than half of the top post by engagements).
  • Mention organizations to add authority to your tweets, especially national societies if they're featured in the content.

Test Your Own Tweets
How Do You A/B Test on Twitter?
A true A/B test requires two messages be run simultaneously and to randomly assigned. Each ad would be delivered to 50% of your target audience. The only way to do that with tweets is to use Twitter Ads.

We, however, took the free-of-charge route of posting multiple tweets for the same article throughout the week, thinking this would yield useful insights about how to improve our tweets. The great frequency of tweets from our account, large size of our audience, and the short half-life of a tweet lends itself to giving you fairly accurate results.

We also tested our hypotheses multiple times over the two months. For example, if we were testing (A)"top tips" vs. (B)"5 tips," we'd run that test during week one. We'd then find a different article to run that same test during another week to see if it delivered the same results.

A/B Testing-Measurement
We first needed a way to measure success. Since two Twitter goals involve sharing useful, shareable health information (reach) and driving traffic (clicks) to our sites, we chose the two measures of engagement, defined as retweets, replies, and favorites combined (these drive reach), and click-throughs.

Try Testing One or All of These

  • Length of a tweet: Do longer or shorter tweets get more engagement or clicks?
  • Image or text only: Does using an image or having only text in a tweet work better?
  • Do you get more website clicks when using your organization's name within the tweet or not?

Share the wisdom and let us know what you tested and learned to make your tweets more effective, either in the comments section or within the SMHN Community.

Written by Makala Johnson and Tony Hart, members of Social and Digital Innovation team for Mayo Clinic.

Choose a message to share 
Want to know your audience better? Try a little A/B testing.
Why are images so important in your social media feeds? Find out here:
Ever carried out any A/B testing on your social media content? Maybe it's time you did.

Tags: Metrics & Analytics, Strategy, Tactics & Best Practices, Twitter

Great explanation of A/B testing. Can you share some examples of what you discovered?


Great explanation of A/B testing. Can you share some examples of what you discovered?

Jump to this post

Definitely Dan.

Some top tips:
1. Short, succinct tweets generally do better than longer tweets
2. Pull info from the page you are sending people to:
2a. If you are looking for clickthroughs: Pull the headline, ask a question, or use teaser text.
2b.If you are tweeting for engagement: Pull a short fact or statistic from the article.


Great explanation of A/B testing. Can you share some examples of what you discovered?

Jump to this post

Thanks, Tony. Great tips!

Please login or register to post a reply.

© Mayo Clinic Social Media Network. All Rights Reserved.