A/B TEST 

(tool)

Brief description

Compares two versions, control A versus variant B, to determine which one works better or which one users prefer.

Quick Guide

  • Identify and determine the element you want to get more information or improve (text, color, functionality, size, pricing, packaging, name, flavor) of your solution and select two variants of that element to compare, this is a recommendation.
  • Define and create your base prototype (Control A).
  • Creates Variant B.
  • Identify your target users and define how many participants you need.
  • Define your measurement method to compare and decide: quantitative (numerical data, e.g. percentages) or qualitative (non-numerical data, e.g. comments).
  • Gather the users and individually or in groups (2-3) present either Control A or Variant B or both at the same time.
  • Give users time to observe and compare.
  • Ask questions or provide a survey so that users can comment or answer which one they prefer.
  • Analyze the responses and, depending on the indicator, make modifications.

Benefits

  • It helps you obtain specific and objective information about what users prefer when comparing two prototypes (value proposition, price, features).
  • It allows you the versatility to use it to complement and reinforce other types of experiments (home page, interview, survey).
  • It allows you to simultaneously test two prototypes in a simple way (they can be radically different or similar).
  • It helps you to solve doubts or assumptions of different elements of the prototype of your solution or concept.

Helpful Tips

  • When testing food products, do not test more than 2 variants to avoid users having too many flavors and not being able to discriminate.
  • In case you select more than one element to evaluate, change one or a few elements. If you change too many at once you will not know what worked.
  • For quantitative tests randomly divide users half with Control A and half with Variant B; for qualitative tests all have Control A and Variant B.

Application

Let’s learn and share your experience with racu