IANteractive Dinner With Kitara Media

Last night Ian Fernando of Infinite FB put on a dinner at The ...

Always Split Test

Split testing is extremely important. I’m going to explain to you why it’s so important, using a real life, raw, very serious example. ...

Affiliate Marketing Terminology

I decided to put together a quick Affiliate Marketing terminology guide for you guys. Please leave a comment if you want any additional terms ...

Always Split Test
Gravatar
Brian

Split testing is extremely important. I’m going to explain to you why it’s so important, using a real life, raw, very serious example. I’m going to show you that there’s always a way to be testing something else, never settle for the industry norm, no matter what you are doing. Realistically this is BARELY split testing, this is only scratching the surface as to how far you should go in your efforts. You could be testing many many more variations on top of the single change I’m showing you in the below examples. But, the article is to get the point across that you should be doing it in the first place, rather than going in depth about it with my volunteer, Kevin.

So, let’s go through an example of an experiment that I did with Keith, a homeless man. First, realize this… This experiment improved this mans earnings by over 100% over several days thanks to me and on top of that I paid him to let me take pictures (which he was very happy about being featured in a blog article online, especially after I explained exactly what I was doing). I truly believe it was not so much about the actual sign in this case and people would have been donating anyways if he REALLY got their attention in some other way (obviously the cardboard wasn’t working) because he was a really cool guy with a great story once I actually talked to him. Nonetheless, on to the story…

Boring. The rate of people giving him money is low. Time for a split test.
Boring. The rate of people giving him money is low. Time for a split test.

As you can see in the first picture, he was using a very standard “I’m homeless, help me” sign with no major call to action. Just bullets of what is wrong with him. The biggest call to action on this sign is “Hi, I’m Keith” which is not really a call to action, and isn’t anything that would really motivate many people to come over on. So, that’s the biggest flaws in his banner.

The next problem was that his cup (goal) of what he wanted people to do , was tucked between his sign between that and his leg and was not very inviting. This would be like putting an opt-in form or buy button on the footer of a lengthy website tucked in the corner blended with the website colors so you can barely see it, etc. You need to call action to what you want someone to do, not tuck it away in the corner.

So now we move on to a split test. The first thing we always do is make slight modifications to what we already have. In this case it wasn’t practical to change the copy on his banner, so we decided to at least make the goal of what he wanted people to do at least a little more appealing, so we asked him to pick up the cup and try that for a while. This didn’t help his conversions much at all.

Still boring, but this time he’s at least holding up the cup…

Next, we’re going to do an actual split test by comparing the results of another choice with the results and information that we already have about how well his current setup works (not very well).

Introduced a higher converting banner w/ bribe, working much better.

What we did here was quite different than what most homeless people would do. We focused on a different angle. We already have the “I’m homeless, help me” stigma attached to people that are sitting on the side of the street with a cup, so we don’t necessary need to make that a prominent part of our banner. The next big difference is that we changed colors and went from cardboard to white to spark the interest of people walking by instead of automatically having negative associations that they have with cardboard and homeless people. We want this to look like a new age homeless man who is really trying to make it work for himself. The biggest difference, is that we are now introducing a bribe. We are basically saying “hey I’m homeless, help me, donate to help feed my family and pay my medical bills… but not only that, if you donate right now I’ll give you a free squirt of hand sanitizer”. This can be applied to list building and all sorts of other aspects of Internet Marketing almost as is (don’t try to bribe people to complete offers though, just bribe them to get on your email list). One solid strategy of building a list is that you get people looking at your opt-in form then give them something for free aka a bribe to get them to pull the trigger and fill out the opt-in form for your email list.

Now he’s smiling, interacting and building relationships with potential customers, even more conversions.

By the way, this experiment wasn’t just for shits and giggles, the homeless man started to draw a crowd VERY quickly on this busy street and people were genuinely intrigued by the fact that a homeless man was giving away free hand sanitizer for donations to help his family/medical bills. I talked to a few people walking by on the street and asked them what was going on (playing stupid) and they were saying how there was this “genius” homeless man who they were happy to give money to because he was showing that he must be sober and really trying to make things happen based on his cool concept. In fact, this may have even have made the local news.

Moral of the story, always be split testing and trying different minor tweaks on your landing pages to try to increase conversions. Things as little as colors can play a major factor. I wrote in black as if a homeless man wrote the sign) then I added an “out of place” kind of irrational element to the equation by using a red cup. In affect, what it did was create a buzz of the people walking by and got them talking about it and debating where he got the cup and if it was a joke or if he was a genius.

The better call to action (interesting) banner with a bribe worked best!

P.S. This BARELY touches the surface of how far you can go with your split tests. The next avenue to explore would to be see how many different variations of the 2nd sign we can do (with the copy moved around and the product switched/moved around etc – there would be many more tests from that point).

P.S.S. This is a visual example from real life that’s meant to show you, hopefully you weren’t offended by it. If you were then you got the wrong message out of this. The homeless man, Keith, was proud and happy to do it. I may bring him in for an interview and do a video on him. He really has an amazing life story. I’d like to feature him in something that can potentially help others avoid his unfortunate situation and put some more money in his pocket to get back on track with his life.

If you enjoyed this, you’ll love our newsletter.

Secrets So Exclusive We Don’t Even Publish Them On Our Blog. They are totally free!


Displaying 96 Comments
  • GravatarDerek
    on March 28, 2010 at 12:48 pm

    Nice job Brian, looks like you picked a good case study to work with. He’s going to be eating well for the next while, begging for cash never works great.

    The ones that do make enough to get by always have a service of some sort and I’m sure you’ve inspired him to think up of methods on his own.

  • GravatarMike Chiasson
    on March 28, 2010 at 2:11 pm

    HAHA this is an epic split test Brian! Granted I’ld be more apt to give money and avoid touching the sanitizer all together but that’s just me!

  • GravatarXPresence.com
    on March 28, 2010 at 7:23 pm

    Thinking outside the box! Loving it! Not only does this example show split testing, but also shows how simple creativity can take you from fitting in to standing out. Whether that’s for a bum or a website, doing it differently is going to draw that crowd.

    Very nice!

  • GravatarEdgar
    on March 29, 2010 at 7:35 am

    Fantastic article Brian! It would be even more interesting if you have included the conversion rate between the two tests.

  • GravatarBrian
    on March 29, 2010 at 8:07 am

    Overall revenue/ROI was over 100% better with the white than the cardboard.

    Conversion rate itself is a bit hard to gauge because we would have to count every person that walked by vs how many of those people stopped and at this extremely busy location it would be next to impossible to count that many people. The point of the article is still there even without the actual conversion rate for the homeless example.

  • Gravatarchris
    on March 29, 2010 at 8:16 am

    did anyone actually use that hand sanitizer? either way great idea id prob give that dude a couple bucks if i saw that sign

  • GravatarJohn Rampton
    on March 29, 2010 at 10:17 am

    I love it… I’m sure people used the sanitizer! Thanks for the great article Brian and for the great visual explanation…

  • GravatarBrian
    on March 29, 2010 at 2:51 pm

    It was somewhere around a 50/50 split of people that actually used it. Some were just so amazed with this guy that they just gave money and took pictures.

  • GravatarAdam
    on March 29, 2010 at 3:54 pm

    Hilarious example of split testing. Even after looking past the obvious humour you provide a a great article on why you should do this testing on your pages.

    I didn’t realize the importance until I tried it on two different pages, the only difference was the call to action button was larger and more centered and the conversion rate more than doubled.

  • GravatarJack
    on March 29, 2010 at 5:50 pm

    A homeless guy in Philadelphia had a sign that said:
    “Lost? Need directions? I can help.” and in small print, below “donations gladly accepted”

    I asked him where to find a pharmacy and a particular office building. Sure enough, he knew without hesitation, even drawing me a map. Saved me a LOT of wandering in near-blizard conditions around so I gave him $10.

  • GravatarMaintenance Man
    on March 29, 2010 at 6:18 pm

    Split testing to help the help the homeless. I like it.

  • GravatarScott Berkun
    on March 29, 2010 at 8:15 pm

    This is great applied design – way more interesting than yet another endlessly ethereal article about theory.

    However, this does show the trap of A vs B testing. The A design had some very basic problems (hard to read, too much text, etc.) – any design that fixed those would have done much better. And simply because B fixes these, some odd and likely negative things come along for the ride.

    For example, I doubt most people would want hand sanitizer from a homeless person compared to less personal things (a blessing/fortune telling, a toy, a newspaper, etc). It’s an odd-match – but you don’t get data on how odd these things are as you’re comparing ALL of B vs. ALL of A, in which case you could offer almost anything in a bribe and it would score better than A.

  • GravatarMichelle Quillin
    on March 29, 2010 at 9:26 pm

    I don’t know what to say! I’m split between, “Ohmyword — you used a homeless man for a marketing test!?” and “Ohmyword — we all learned something really special here, and so did the homeless guy!”

    Gutsy. That’s what this was. Gutsy. I’m so glad you had the chutzpah to do this. What a great example of split testing, and such a unique way to do it. Love it!

    Michelle Quillin for New England Multimedia
    http://twitter.com/nemultimedia

  • GravatarEd Kennedy
    on March 29, 2010 at 9:45 pm

    Thanks for bringing the human element in to marketing so elegantly. I’m posting a link to this article on my blog!

  • GravatarBrian
    on March 29, 2010 at 10:52 pm

    Scott- I’m on the same page as you Scott. There is tons of additional split testing to do if we really wanted to keep on increasing his conversions, this is not a solution but more-so showing how to begin the process for Internet Marketers, I didn’t want to go too in depth with the homeless example and have thousands of people on the street with fancy signs trying to take advantage of it ;)

    Michelle- I actually explained to the homeless man (Keith) exactly what I was doing and even told him what I was thinking of using for a title. He was all for it and said that he hoped people could learn from this. It was a totally amazing experience and although I used the subject to drum up some buzz if you read the entire post I’m very sincere about not mocking or making fun of Keith, the homeless man.

  • GravatarMichelle Quillin
    on March 29, 2010 at 11:26 pm

    Brian, I did read the whole thing. You were very humble and gracious, and obviously cognizant of his position. Your kind attitude is what made it so gutsy — I think someone who didn’t care about Kevin’s position, but only about their own interests, wouldn’t need courage to approach him.

    I think this is so cool.

    Michelle for New England Multimedia
    http://twitter.com/nemultimedia

  • GravatarCTRtard
    on March 30, 2010 at 12:10 am

    Great post idea. I often think about real world split testing when I’m out and about. But it never occurred to me when looking a the signs homeless guys hold up. But you made one glaring error: you didn’t give the guy a clipboard so he could build his list. Dude, the money is in the list!

  • GravatarBrian
    on March 30, 2010 at 12:26 am

    :) I actually talked to him for a good hour after we were all done and he was up for doing some more testing. But, I told him that it wasn’t about getting too deep that I just wanted to get the main point across. Marketers would be able to take that and learn from there how to continue the process and branch it out within their niche. I’m thinking about doing a video with him to get his story out there and hopefully help him.

  • GravatarAaron Dwyer
    on March 30, 2010 at 5:39 am

    Great post, really enjoyed it. As I pass the homeless people and the buskers in the city when I sometimes venture in there, I often think about redoing their signs to give them more business.

    If a busker has a CD, I’ll buy it and chat with them, but they always glaze over when I mention how making a change will get them more sales. I don’t know why.

    Aaron

  • Gravatar@deepakaujla
    on March 30, 2010 at 6:07 am

    Always thought the most rewarding pieces of work are those that help others and you managed to do this using split testing – awesome! Great job Brian.

    Completely agree with “never settle for the industry norm”. If anything, most successors within industries have shown us that by being creative and engaging in a helpful way, helps you to step ahead of the competition. In this case Keith with the creative white board stepped ahead of Keith with the cardboard.

    Keith, if you ever read this, thanks for contributing to the community and good luck for the future!

  • Gravatarrinkjustice
    on March 30, 2010 at 7:33 am

    Bravo Brian. Compassion and creativity can really make the world better.

    This should be a clarion call to all marketers: let’s get our hustle on and make someone’s life better using our talents.

  • GravatarJay
    on March 30, 2010 at 11:47 am

    I think that’s excellent Brian.

    I can’t think of a better way to help someone while teaching at the same time.

    Bravo!

    Jay

  • GravatarJanis
    on March 30, 2010 at 12:30 pm

    This is great. I’ve never thought of this, but it’s an excellent way how online world methodologies can be applied to offline – such as this case. And with success!

  • GravatarJohn Joe
    on March 30, 2010 at 1:56 pm

    I like it! Keep ideas like this coming.

  • GravatarJoe Zeigler
    on March 30, 2010 at 5:27 pm

    This is thought-provoking on so many levels – marketing, advertising, anthropological, sociological,… Well done.

    And instead of just giving a man a fish, you taught him how to fish better.

    Well done.

  • GravatarJosiah Staggs
    on March 30, 2010 at 5:37 pm

    @deepakaujla: I think if you would try the “What if i told you…” approach they may be more interested.. I guess that’s another thing to test hah.

  • GravatarJoe
    on March 30, 2010 at 5:47 pm

    Ha ha ha this is a great article and insightful. I am clueless about marketing. Too bad I can’t give away free hand sanitizer on my website to boost traffic.

  • GravatarLeslie Lynn
    on March 30, 2010 at 6:11 pm

    Awesome.

  • GravatarRay
    on March 30, 2010 at 10:17 pm

    I love this. Really got my mind going too as far as how I want to build my squeeze pages. I can visualize how my web pages should look more easily now.

    Thanks,

    Ray

  • GravatarErin
    on March 30, 2010 at 10:31 pm

    Love it. And this is beside the point but driving me crazy: the hand sanitizer and cup should be at the top for easier access and less proximity to gross city sidewalk. I know, I know, you would have done further testing… I just can’t get past the idea of bending down to get at the hand sanitizer.

    • GravatarBrian
      on March 30, 2010 at 10:43 pm

      @Erin – Haha, yah. I agree with you. Actually, it was just on the ground for the pictures mostly, he was walking around with it and holding it up when people came over. I wanted to make the point that nothing should ever be considered perfect and you should always do tests on top of tests hence the randomness on the white one. If the banner was a website that you couldn’t hold up in the air (as he was able to with the banner), then the button/opt-in form would need to be above the fold/higher up in most cases.

  • GravatarHenryb
    on March 31, 2010 at 1:59 am

    When the student is ready the teacher will appear. You have given us
    a powerful example of split testing, just what I needed for my next challenge. Whilst I was doing split testing at craft-markets with our step stool cows, it did not quite sink in that the same can be done on line. Thank-you for jogging my brain.

  • GravatarMelissa Thompson
    on March 31, 2010 at 11:10 am

    This is awesome.
    You have done an outstanding job of teaching a man to fish…
    and helping others by the example. What a concept!
    If Keith put up his story online, his cup would be overflowing.
    Awesome, just awesome.
    I am putting a link on my blog…

  • Gravatarjustaguy
    on March 31, 2010 at 11:17 am

    Excellent post just love it and it shows how really important is split testing.
    However… did you test the source traffic (:
    did you tried the boring sign in near the wallstreart “traffic” v.s. the bus station “traffic” ?
    conversion rate alone doesn’t tell the whole story.
    you may get last people donating in wallstreat but more money…

    Love this post , well done!!!

    • GravatarBrian
      on March 31, 2010 at 11:21 am

      @justaguy – Lol, ya there are many more possible ways and variables to test this out from here. Many, many, many. This is only the beginning in what you guys should be doing in your online testing. This is only step 1.

  • GravatarEd
    on March 31, 2010 at 11:47 am

    This is the most creative split test that I have seen.
    Everyone wants something for a donation, even if it’s just a good feeling but hey..you’re in the city and your helping out the needy and keeping clean at the same time.
    I’m sure people will go out of their way to toss some money in his cup.
    Great job!

  • GravatarCamille
    on March 31, 2010 at 11:54 am

    this is my 1st time on your site, and loving it already! i’ve read so many articles on split-testing, but this really shines a fresh light on the topic. thanks!

  • GravatarJeremy Aragon
    on April 2, 2010 at 3:19 pm

    DUDe! You had me at homeless! I just read the post by following a vaynermedia link and am now a fan of whoever you are! I read perry marshals definitive guide to adwords a few years ago and really grasped the concept of split testing but the way you demonstrated your skills in real life…kudos bro!

  • GravatarShane
    on April 2, 2010 at 3:43 pm

    Dude, this is such a brilliant idea for an article! Really awesome stuff and an instant subscribe for me.

    Cheers,
    Shane

  • GravatarJustin
    on April 4, 2010 at 3:21 pm

    I ran an a/b test on Pickfu. People do like the hand sanitizer sign, but it’s fun to see the comments for the people who voted for the original sign.

  • GravatarSteve
    on April 5, 2010 at 1:36 am

    There was a homeless guy who hung out near the Vanderbilt Campus in the late 90s. He would solicit drunk students for cash in exchange for a verse of rap about whatever you requested(He had even less lyrical talent then shame). He understood how to exploit race and socioeconomic anxiety better then any marketing consultant I’ve ever met. I gave him at least $200.00 over 2 years.

  • Gravatarjr
    on April 5, 2010 at 12:26 pm

    Great Article Brian. Thanks For Sharing.

  • GravatarJay Willingham - Campusbyte
    on April 5, 2010 at 4:37 pm

    This was an excellent split test. It’s amazing how you can apply the concepts we use online to everyday life with often similar results.

    Great article.

  • GravatarJewishAtheist
    on April 7, 2010 at 1:26 pm

    Genius!

  • GravatarRex Dixon
    on April 7, 2010 at 3:42 pm

    Love this! Quite an interesting split test!

  • GravatarJ.P.
    on April 7, 2010 at 8:26 pm

    So many things!

    * Amazing way to show the point.

    * I also truly think that you taught Keith something.

    * I don’t know if the public is new every day or he has a lot of repeat visitors.

    What do you think will happen once the novelty washes out?

    Maybe could be a good idea for the guy to come up with a new sign every week?

    I know a greengrocery where the guy does amazingly funny signs for the merchandise (in a blackboard). And is always a conversation starter.

    And last: Cool link bait piece mate! Maybe a curiosity headline could drive more in Social Media/bookmarking sites ;-)

    Make a video with this case study, this guy story and some specifics and you have another hit!

    you got yourself another subscriber today :)

  • GravatarMatthew Loop
    on April 9, 2010 at 12:03 am

    That’s awesome, Brian! I really do think homeless people should split test more. It’s all about maximizing effectiveness either on the streets or online. Great example :)

  • GravatarRyan
    on April 10, 2010 at 1:13 am

    My favorite part is how you put your logo onto his jacket in all of the pics, and made it look like it was supposed to be there, too!

  • GravatarDale
    on April 12, 2010 at 9:27 am

    Great job Brian! i like the way you and Keith worked together to pull this off. The real life example of split testing is just Incredible! And I just could not help to notice how two people (Brian and Keith) came together to create such a buzz!

    No one knows how far Keith will go from here. You have taught us all a very valuable lesson and how it could be applied to our business as well as our life!

    Thanks Brian.

    Keith, if you are reading this. Thanks! It’s people like you and Brian who makes a difference.

  • GravatarJeroen
    on April 13, 2010 at 10:41 am

    Some great example of ‘thinking out of the box’ here ! :-)

  • Gravatarrobert
    on April 14, 2010 at 3:18 pm

    i like the way you and Keith worked together good job

  • GravatarCharlene Jaszewski
    on April 18, 2010 at 6:15 pm

    Brilliant. This is making me want to makeover all the homeless signs I see in the subway! Humor works!
    I don’t know why but there are more “creative” homeless signs in San Francisco. Here are some I saw:
    *Help! Family kidnapped by ninjas, need $$ for karate lessons.
    *Can you help me with a deposit on a cheeseburger?

    and my favorite:
    *Bet you can’t hit me with a quarter! (seen at an intersection)

  • GravatarDavid Jaeger
    on April 19, 2010 at 12:43 am

    I love it!!!

    Taking online “abstract stuff” and making it real-world. You’ve got me inspired…

  • GravatarIshan Ghosh
    on April 19, 2010 at 12:48 pm

    I’m curious to know why the original call to action wasn’t changed if the new board was going to brought in anyway.

    Reminds me of the classic ‘What is Advertising’ story of the blind man who, much like your guy, had a banner around his neck saying, ‘BLIND’ and wasn’t getting any where until some kind advertising soul, changed the message to, “It’s spring and I’m blind!”. Guess what happened?

  • GravatarRob Shields
    on April 20, 2010 at 8:59 pm

    Love it (like everyone else)!

    I am curious as to the ROI (while less important in a lot of A/B testing, it seems more relevant here):
    When you say 100% improvement in ROI, is that actually “100% improvement in revenue”, or did you factor in the cost of the white paper (cardboard is easily found, but white paper of that size probably isn’t), cup (could be more readily found than then paper, admittedly), duct tape, and sanitizer?

    The new sign may draw in more money, but it may be less effective from an ROI POV if the cost (amortized over all the uses) was too high, in which case it wouldn’t be a success.

    This is all quibbling, I know, and mostly beside the well made point you were getting at, but we need to be careful about optimizing conversions or revenue and not net dollars (if that’s the goal).

    Could you post the before and after dollars? I’d love to play with them ;)

  • GravatarGurpreet Singh
    on April 21, 2010 at 1:32 am

    A practical example is always easy to understand! Thanks for the explanation of experimenting.

  • GravatarKristof
    on May 8, 2010 at 11:33 am

    This is seriously awesome dude.

  • GravatarMichael Aagaard
    on May 20, 2010 at 5:14 am

    This is the coolest thing I’ve ever seen! I’ve forwarded it to practically everyone I know…

  • GravatarDavid Damron
    on May 31, 2010 at 2:09 pm

    I’m retarded. Okay…I am not retarded (no offense retarded folk), but I have always been SCARED of split testing. Not because I don’t want to succeed, but because I am afraid of screwing something up. In all actuality, by not split testing, I am screwing up a TON more than if I were to be split testing.

    Thanks for sharing such a simple but obvious example. Sometimes, we all need something to relate better too and this is just that.

    Very much appreciated.

    David Damron
    The Minimalist Path

  • GravatarAyça
    on May 31, 2010 at 4:58 pm

    I am looking for an example for sign test and ı found that.So,that’s really fun :) :) :)
    Thank you…

  • GravatarAndrea Kropp
    on July 20, 2010 at 11:01 am

    Loved the photos. Hope its OK to paraphrase this story in an upcoming class. The visuals are SO powerful. First time visitor, but I’ll be back. Andrea

  • Gravatarshecky
    on July 25, 2010 at 12:03 pm

    Great way to take an everyday example, and use it for a powerful lesson. Nicely done, Brian.

  • GravatarKathy
    on August 6, 2010 at 3:53 pm

    Oh, wow, Brian – This was just amazing! First of all, I’m a very visual learner, so this example explained split-testing in a way I could really understand. Thank you so much for that!

    Also, I am truly moved at how you worked with Keith – I would love to see his story unfold. Undoubtedly, you were a blessing not only to him, and not only to those of us trying to learn marketing techniques, but also to the many people who passed by on the street that day. Who knows all the good you’ve done!

    Thank you again! I’m so impressed.
    Warmly,
    Kath


Please note: Comment moderation is enabled and may delay your comment. There is no need to resubmit your comment.