Where Problem Interviews Break at Scale (AI SaaS)

Problem interviews are wonderful at small scale. Past about thirty calls, AI SaaS practice has its own break patterns. Here is what fails and how to keep the practice useful.

The Synthesis Bottleneck

At ten interviews, you can hold the trust boundaries in your head. At fifty, you cannot. Notes pile up faster than synthesis. Patterns get buried.

The fix is to synthesize after every five interviews. Treat it as a recurring chore, not a project finale.

The No-AI-Frame Discipline Decays

Around interview twenty, founders start sliding. They have heard the patterns. They cannot help asking the AI question explicitly. The framing creeps back in.

This is when data quality drops. Late-round calls become AI-position surveys instead of workflow discoveries. The fix is to treat the discipline as more important the further you get, not less.

Audience Drift

At small scale, you talked to a tight audience. As you scale, you say yes to anyone willing to chat. By interview forty, your sample is broader than at interview ten.

The fix is to keep a written audience definition and reject interviewees who do not fit, even when it feels rude. Your data is only as good as your sample consistency.

Trust Boundary Confusion

Specific to AI SaaS. As you scale, you talk to interviewees with different trust boundaries. The patterns blur. You can no longer tell whether the boundary you should respect is the one ten of fifteen interviewees described, or whether you have accidentally averaged across two segments with different boundaries.

The fix is to segment your synthesis by audience filter. Two segments, two trust boundaries, two product specs - not one averaged spec that respects neither.

The Eval-Framework Trap

AI SaaS founders are particularly susceptible. Around interview twenty-five, the engineering brain wants to introduce eval infrastructure. This sounds productive and is mostly procrastination. By the time the eval is set up, the patterns it would have measured were already obvious from notes.

The fix is to delay eval infrastructure until usage data exists. Eval against real failure cases customers surface, not against synthetic test sets you invented.

Note Quality Erodes

The first ten interviews were energizing. By thirty, notes become shorter. Quotes get paraphrased. The vivid AI failure stories get reduced to one-line summaries.

The fix is to lower the bar of what counts as "done." Three direct quotes, one trigger event, one trust boundary location, one surprise. Minimum.

Co-Founder Sync Problem

At ten interviews, both founders attended most. At fifty, only one attends each. Each carries different vocabulary, different patterns. The team gets out of sync.

The fix is a weekly fifteen-minute sync where the most interesting AI failure story of the week is talked through. Verbal recap creates shared instinct that written notes do not.

Pattern Saturation Becomes Confusion

You hit a wall around interview forty where you can predict what the next call will say. This is saturation, which is good. But it can feel like confusion because the patterns have stopped surprising you.

The fix is to either stop and ship, or rotate to a new audience. Do not keep interviewing the same segment hoping for new insight. The next dollar is in building or in a different segment.

The Bigger Pattern

At small scale, problem interviews are a tool. At scale, they become an operating practice with supporting habits: synthesis discipline, audience filter consistency, no-AI-frame, trust boundary segmentation, restraint about eval infrastructure. AI SaaS teams that hold these habits stay calibrated. The ones that do not let the practice degrade until it stops producing signal.