Before we rebuilt The ENTERTAINER's pricing model, there was a version of this project that ran as a survey (a clean, well-designed questionnaire sent to thousands of users, asking how they felt about subscriptions versus one-off purchases). It came back with tidy numbers. It also came back almost useless, because it told us what people said they'd prefer, not what was actually stopping them from paying.
What the 40 interviews found that the 4,000 responses didn't
The survey told us people were "neutral to positive" on subscriptions. The interviews told us why that number was so soft: people weren't evaluating the subscription price, they were doing quiet mental math about what happened if they lapsed and lost access to a deal they'd come to rely on. That fear doesn't show up as a survey answer, because nobody writes "I'm afraid of losing something I don't have yet" on a five-point scale. It shows up as hesitation in someone's voice on a call, followed by a specific, concrete story about the one time they got burned by a different subscription somewhere else.
A survey measures the opinion people are willing to hand you in writing. An interview finds the specific story they're reasoning from, and the story is almost always the actual input to the decision.
Where I still reach for a survey anyway
This isn't an argument that interviews always win. Surveys are the right tool once you already know the shape of the problem and need to know its size (how many people feel this way, which segment feels it most). The mistake is reaching for a survey to find the problem in the first place. Use interviews to discover what question is even worth asking at scale; use the survey to find out how widely the answer you found actually applies.
What makes an interview actually work
- Ask about the last specific time, not the general opinion. "Walk me through the last time you almost cancelled" beats "how do you feel about cancelling."
- Let silence sit. The real answer is usually the one that comes after the polished one, once the obvious response has already been said.
- Interview until you stop learning, not until you hit a number. Forty wasn't a target we set in advance, it's roughly when new interviews started confirming the same handful of patterns instead of surfacing new ones.
The insight that actually moved paid conversion from 13% to 26% (that the fear was about losing access, not about price) came from interviews. No survey question we drafted, before or after, would have produced that sentence.
The honest trade-off
Interviews don't scale, they're slow, and they're uncomfortable to sit through when someone describes exactly how your product let them down. That discomfort is the signal a survey sands away before it ever reaches you. For the decisions that actually move a number, I'll take forty uncomfortable conversations over four thousand comfortable ones, every time.