You have eight users who agreed to talk to you. Lovely. Now open your calendar and try to find eight one-hour slots that also work for them — across three time zones, around your sprint, around their jobs. By the time the last conversation happens, three weeks have passed, one person never showed up, and the thing you wanted to learn about has already shipped.
That, in one paragraph, is why founders ask about async interviews. Not because live conversations do not work. They work brilliantly. The problem is that the logistics around them quietly eat the habit.
So let’s walk through the real trade-offs: what live interviews do that nothing else can, what async interviews do that live ones cannot, and how to decide which to run for the question in front of you.
First, what we actually mean
A live interview — researchers usually say moderated — is a scheduled conversation in real time: a video call, a phone call, or coffee. You ask, they answer, you follow the thread wherever it goes. Nielsen Norman Group defines moderated remote testing as researcher and participant sharing the same virtual space at the same time.
An async interview — usually unmoderated in the literature — is one the participant completes on their own schedule, with no human researcher on the other end. The questions arrive through software: a structured questionnaire flow, a recorded-video prompt, or, increasingly, an AI interviewer that asks follow-up questions in a conversation. As NN/g’s Kathryn Whitenton puts it, removing direct interaction between researcher and participant is both the core benefit and the core drawback.
That sentence is the whole article. Everything below is just working out which side of the trade-off applies this week.
What live interviews do better
They follow the person, not the plan. The best moments in an interview are the ones you did not script. A participant mentions, in passing, that they export everything to a spreadsheet before doing the “real work” — and a good interviewer drops the guide and asks them to walk through that spreadsheet. In NN/g’s study of AI interviewers, this was precisely the gap: automated moderators can probe when an answer is short, but they do not reliably chase unexpected threads, skip a question that has stopped making sense, or reframe one that is landing badly.
They read the room. A pause, a half-laugh, a glance away — these tell you when “it’s fine” means it is not fine. Rob Fitzpatrick built The Mom Test around this problem: people politely tell founders what they want to hear. A skilled human can notice the politeness and gently dig underneath it in the moment. Software, today, mostly cannot.
They create social commitment. When someone knows a real person set aside an hour for them, they engage more seriously. NN/g notes that in moderated studies, social pressure can compensate for low personal motivation: participants try harder because someone is watching. Async participants, alone with a screen, can drift, rush, or multitask.
They build the founder’s own ear. This one rarely makes the comparison tables, but it matters. Doing your first ten interviews yourself teaches you how users actually talk — the words they use, the workarounds they are embarrassed about, the difference between a real complaint and a shrug. That education does not transfer from a transcript quite the same way.
So if live interviews are this good, why would anyone do anything else?
What async interviews do better
They delete the calendar problem. No scheduling, no time zones, no waiting for a mutual slot. The participant answers on Tuesday at 23:40 because that is when their toddler is asleep. This is not a small convenience; it is often the difference between research happening and not happening.
Across more than 14,000 moderated studies analysed by MeasuringU, about 8% of confirmed participants did not show up, rising to 10.4% for remote B2B sessions. That is exactly the audience many B2B SaaS founders are trying to reach. The standard advice is to over-recruit, pay appropriately, confirm, remind, and make rescheduling easy. All sensible. All extra work. An async study has no empty chair. Someone who does not respond costs you an invitation, not a calendar slot.
They scale past your own hours. A live interview costs the hour, plus prep, notes, and synthesis. Eight interviews can consume a working week. Unmoderated research still has costs — setup, pilots, quality checks, analysis — but it can run in parallel while you write code. NN/g’s cost comparison makes the same broad point: unmoderated studies can lower researcher time and cost, though sometimes at the expense of finding quality. For a founder, that trade-off is real. Calendar time is the constraint that usually breaks the habit.
They can make honesty easier. People disclose more when no human is watching. This is one of the oldest findings in survey methodology: Tourangeau and Yan’s review of sensitive questions shows that self-administration can increase reporting of embarrassing or socially awkward truths. For a founder, “your onboarding confused me and I gave up” is exactly such a truth. Saying it to your face is hard. Typing it at midnight is easier.
Keep the caveat. A later meta-analysis by Gnambs and Kaspar found the disclosure effect is real but smaller and more conditional than early enthusiasm suggested. Async can help honesty; it does not guarantee it.
They ask everyone the same questions. When thirty participants get the same questions in the same order, comparing answers is easier. Live interviews drift. By interview six, you have unconsciously rewritten half the guide. For discovery that drift can be useful. For structured feedback, it is often noise.
The honest costs, side by side
Live interviews tax your calendar, cap your sample at however many hours you can spare, and put a founder — the least neutral person alive — in the moderator’s chair. We have written before about why founders struggle to moderate their own interviews: you flinch at criticism of the thing you built, and participants sense it.
Async interviews tax your preparation instead. Because nobody can rescue a confusing question in the moment, NN/g warns that unmoderated studies need more meticulous planning than moderated ones. A badly worded prompt quietly ruins thirty responses instead of one. You will also meet less engaged participants, the occasional professional tester, and answers you wish you could have probed one layer deeper.
There is an analysis cost too. Thirty async conversations produce thirty transcripts, and transcripts do not read themselves. NN/g notes that unmoderated studies accumulate data quickly, and for qualitative recordings you still have to review what people said. Plan for synthesis before you launch the study, not after the responses pile up. Modern tooling helps, but you should still read the strongest and strangest transcripts yourself.
And the conversational quality of automated moderation is still uneven. In NN/g’s study, only three of ten participants agreed the AI-led conversation felt natural, and session lengths varied widely. Participants felt heard when the interviewer summarised their answers, but they missed having something human to react to. Anyone selling you async interviews as a finished replacement for every human conversation is overselling, and we build an AI interviewer for a living.
Where AI-moderated interviews actually sit
It is worth separating old-style async — form-like flows, recorded video prompts — from conversational AI interviews, because the evidence base is different.
The most interesting academic result so far comes from economists Felix Chopra and Ingar Haaland, who had an AI interviewer conduct 381 text-based qualitative interviews about why people do not invest in the stock market. The interviews surfaced rich reasoning, and a follow-up showed the interview data predicted participants’ later economic behaviour. That matters because it addresses the standard worry about async research: that you are collecting cheap talk. Talk that predicts behaviour later is not cheap.
NN/g’s assessment, sceptical as it is about discovery work, lands on four use cases where AI moderation already earns its place: structured product feedback, teams with no researchers, interviews in languages you do not speak, and participant screening. Notice what these have in common. In each, you mostly know what you need to ask. The AI’s job is to ask it well, follow up on thin answers, and do it many times without getting tired.
Where AI moderation is not yet the right tool is early discovery in a problem space you do not understand, where the whole point is noticing the thread you did not know to look for. For that, there is still no substitute for a human who knows the domain, sitting with another human, with time to wander.
Writing questions that survive without you
If you run an async study, the question wording carries the whole thing. There is no moderator to rescue a confusing prompt, rephrase on the fly, or notice that everyone is misreading question four. NN/g’s advice for unmoderated studies is blunt: task and question writing is where many studies fail, and a pilot test with one or two real participants before launch is essential.
Three habits make async questions sturdier.
First, anchor every question to a specific past event. “Walk me through the last time you exported data from our product — what were you trying to do with it afterwards?” survives without a moderator. “How do you feel about our export feature?” invites the polite shrug that Fitzpatrick’s Mom Test exists to prevent. Past specifics give a follow-up system something concrete to dig into; opinions give it fog.
Second, tell the participant where the conversation is going and how long it will take. In NN/g’s AI-interviewer study, participants who were dropped into an interview without a proper introduction held back information because they had not yet decided whether to trust the thing asking. A two-sentence opening — who is asking, why, what happens to the answers — costs nothing and changes what people are willing to say.
Third, pilot the study on one colleague and one real user before sending it to forty people. A misread question in a live interview wastes a minute; the same question in an async study wastes the whole sample. The hour you spend piloting is the cheapest research time you will buy all month. For more on wording, pair this with Maren’s guide to avoiding leading questions.
A decision rule you can actually use
Here is the rule I would give a technical founder with 50 to 1,000 users and no research team.
Match the format to how much you know. If you cannot yet write the ten questions that matter — you are exploring a fuzzy problem, a confusing churn pattern, or a market you do not understand — do it live, yourself or with a co-founder, for five to eight conversations. If you can write the ten questions — feature feedback, churn reasons, pricing context, onboarding friction — async will get you more answers, faster, from people who would never book a call.
Use live to find the questions, async to widen the evidence. The formats compound. Three live conversations teach you what the real themes might be; an async study across forty users tells you which themes are common and which were one person’s bad Tuesday. Running the wide check costs you almost none of your own calendar time, which is what makes it survivable alongside shipping.
Protect the cadence above all. Teresa Torres frames continuous discovery as weekly contact with customers, and her advice on getting started is bracingly practical: “step one is just get someone in the door every week”. For a two-person company, a weekly live interview habit collapses the first time a launch week arrives. A realistic version: async conversations running continuously in the background, plus one live conversation whenever the async findings surface something you do not understand.
A worked example: suppose churn ticked up last month and you do not know why. You cannot yet write the ten questions, so the first move is live: three or four conversations with recently churned customers, moderated by whoever did not build the feature in question. Suppose those calls all point at onboarding. Now you can write the questions — which step, what they expected, what they did instead — so the second move is async: the same structured conversation with every user who signed up in the last quarter and went quiet. Live found the thread; async measured how often it appears. Neither format alone would have done both jobs well.
Keep it casual either way. Fitzpatrick’s deeper point is that learning does not require a formal meeting. A good five-minute conversation about a specific past incident beats an hour of hypotheticals. That principle transfers cleanly to async: ask about the last time something happened, not whether they would use a feature. The format changes. The discipline does not.
The comparison, condensed
Live interviews give you deeper follow-ups, real rapport, and adaptation in the moment. You pay for that with calendar time, small samples, no-shows, and your own bias in the moderator’s chair.
Async interviews give you no scheduling, parallel scale, more comfortable honesty on awkward topics, and consistent questions. You pay for that with shallower probing, heavier up-front planning, and more uneven participant engagement.
AI-moderated conversations sit between the two: real follow-up questions at async scale, already credible for structured learning, not yet credible for the most open-ended discovery.
The trap is not choosing the wrong format. It is letting the friction of the noble format mean no interviews happen at all. Three weeks of silence while you wait for calendars to align teaches you nothing. An imperfect conversation this week teaches you something you can act on by Friday.
Talk to your users in whatever format you will actually sustain. When the format stops being the bottleneck, you may be surprised how much they have been waiting to tell you.