How to use Google Calendar to track your A/B testing schedule.

Despite a good variety of A/B testing services like Optimizely, Mixpanel, Adobe Target and all kinds of blends of home grown testing frameworks, companies have always struggled to have a comprehensive calendar which would cover ongoing and past tests.

Poor awareness of which tests are running, along with lack of easy to read historical information on what has been tested, can lead to having to restart testing, throw data away and waste time on inconclusive results.

Ever since I first started testing with Offermatica back in 2007 I’ve tried many different ways to keep my team and the organization aware of what we’re testing and I have to say that for the past few years I found Google Calendar to be the best tool for the job.

Why use Google Calendar?

1. Very quick at-a-glance view of tests running
2. Easily shared with anyone
3. Low overhead

Here’s an example of what a weekly view of testing can look like…

A lot of companies have moved from clunky Outlook to Google Calendar, so why create overhead of managing another calendar somewhere else? Sure it’s not automatically generated and you will have to maintain it, but I’ll show you that setting up the calendar is a matter of minutes and a good standardized naming convention will go a long way.

Setting up Google Calendar for A/B testing.

Step 1: Create a new calendar

Call it something like “A/B Tests”.

Step 2: Create an event.

Make sure to mark the event “All Day” and place the end date some time in the future.

To name the event I usually go with Test Name (##%) where you can update ## with the actual number of users receiving new candidates.

For description you can briefly go over what you’re testing and what the candidates are. Sometime I also put links to PRDs or mockups so other folks can easily understand what is being tested.

Step 3: Share with team.

This is one of the main reasons why we’re setting this up in Google Calendar. It’s very easy to share this calendar with anyone on your team.

Step 4: Maintain and update.

When you make any changes to the test, like removing a candidate or bumping up percentage, you should get into the event and update it with date and change description. Once the test is called, the event title gets an update with Win or Loss and percentage goes to 100% or 0%. I try to put some relevant information about the results too.

That’s it. The important part will be making sure that you maintain the calendar and encourage your team members to use it. If you get asked about what tests are running or even what current percentages are, you should remind folks to go and check the calendar.

Remember that this is just as useful for the rest of your company as it is for you. Wouldn’t it be great to onboard a new PM and just share this calendar with them so they can quickly see what kind of tests have been running. Or maybe there was some issue with metrics being down during certain week — easy, start your investigation by looking at the tests surrounding the date.

Last thing I ask is that you do not waste resource on building your own A/B calendar front-ends, at least not until you try maintaining one of these. Happy testing!