Gimme an A! (some thoughts on A/B testing)

Some thoughts on A/B testing. 

When I get to expound on the same topic 3 times in one morning, it’s time to “blog it out“.

A/B testing. Optimization. Google Experiments. Site improvements. Script improvements. Sales material improvements.

When I hear these things my mind drifts back to whet Deming said all those years ago, “In God we trust, all others bring data.”

The goal is to quit guessing and start learning from our actual users. “I read that green ‘buy it now’ buttons are the best,” is not a test.

Let’s start this way: decide what results you’re after (you know, where the needle is going to pivot) and then do nothing.

That’s right, do nothing. 

We’re going to start collecting data and viewing it in a new light because it’s going to help us establish the A in our A/B test. I know, you’re going to tell me that we can go backwards in Analytics and figure out what the A is but I’m going to stand firm. Let’s watch what happens once we decide what we want to change.

It’s not easy but it’s what works best.

Trust me, even with the old data at your fingertips, looking at what you’re trying to improve after you’ve decided what it will look like when improved is like being in a room lit with a red light when someone comes in and hits the regular light switch. All sorts of details are going to emerge that you didn’t notice before.

That’s my advice. When it comes to A/B testing, “Gimme an A!” You’ll save time and money in the long run.

Good stuff.