Thumb through your 1999 Guide to E-Dominance, and you’ll find a recipe for product prototyping that looks something like this:
1. Develop a minimally viable product.
2. Ask users to test it.
3. Review user comments and adjust accordingly.
This scientific method-inspired model worked until early adopters became tech-savvy enough to realize that lazy marketers were wasting their time, after which beta tests began to piss them off. That’s according to Aha CEO and cofounder Brian de Haaff (@bdehaaff), a Silicon Valley veteran who’s led product launches at Citrix Systems among other prominent software companies. He says minimally viable is no longer OK. For a beta test to yield useful feedback, the product must be lovable — though certainly not flawless — right from the start.
Here are de Haaff’s rules for running better beta tests.
Rule 1: Recruit Ridiculously Engaged Beta Testers
“I listened closely to potential beta customers, and if either there was no excitement or I heard phrases like, ‘Sure, I would like to play with it,’ I humbly suggested that the timing might not be right and we should connect at a later time,” writes de Haaff in the post “Get Beta Programs — a Contrarian View” on his company’s blog. “The potential beta customer is not always right, and it’s actually good for you to say no. It’s important to say ‘no,’ because great products have a driving vision and who and what are left out [are] just as important as what goes in.”
Rule 2: Deliver a Minimum Lovable Product
“‘If you release a product and are not embarrassed by it, you waited too long.’ How many times have you heard or read this? We think the reverse is actually true,” de Haaff writes. “If your goal is feedback during beta, people actually need to use [the product], and that means they need to sacrifice to make the time. And people only sacrifice if what’s new excites them. A shitty beta product excites no one.”
Rule 3: Conduct Pre-Beta Interviews
“Before the (Aha) beta even began, we received nearly 50 hours of priceless feedback, which led to important changes and unique enhancements,” de Haaff writes. “It’s paying off as folks are visibly excited when they see a demo and consider starting to use Aha. We have also created meaningful customer relationships that humanize what we are doing and add personality to the customer’s beta experience. . . . It’s amazing to consider that all of that happened without having a single customer using the application.”
One aspect of beta testing hasn’t changed over the years: “Customers lie. Not maliciously, but they lie,” notes Scott Anthony, managing partner at Innosight, in an HBR Blog Network post titled “Four Simple Low-Resolution Innovation Tests.” Anthony (@ScottDAnthony) recommends four ways to gauge whether your customers will actually pay for your finished products. Our favorite: Before production begins, create and present mock-ups to a small group of customers. Afterward, ask whether they’d be willing to pay a nominal fee to participate in a private beta test when the offering is ready. If they say they’re excited but hesitate over the fee, you know your concept needs work.