# Conducting User Interviews: What Academia and Industry Share and Where They Diverge

- Author: Shuichiro Ogawa (Design Researcher / Design Consultant), https://jamjet-iqbal.com/
- Published: 2026-10-31; updated: 2026-11-01
- Canonical (HTML): https://jamjet-iqbal.com/en/notes/design-user-interview-practice/
- Japanese version: https://jamjet-iqbal.com/notes/design-user-interview-practice/ (Markdown: https://jamjet-iqbal.com/notes/design-user-interview-practice.md)
- Language: en
- Keywords: user-interview, user-research, qualitative-interview, contextual-inquiry, critical-incident-technique, saturation, information-power, participant-response-bias, rapid-ethnography, service-design, customer-discovery, lean-startup, jobs-to-be-done, research-democratization, research-ethics, synthetic-users, ai-interviewer, research-practice-gap

> This note examines user interviewing for making design outcomes, from UIs to services and businesses. It draws on 32 academic works, 14 guidance documents from public bodies and codes of research ethics, 12 materials from research firms and consultancies, and 13 first-hand accounts by practitioners, and checks them against the full texts wherever these could be obtained. The conventions of asking are nearly the same on both sides: ask about specific past events instead of opinions or predictions of the future, do not lead, listen in silence, and read what people say together with observation. Academia has measured the limits of self-report and how strongly the interviewer shifts the answers (in Dell et al. 2012, participants were about 2.5 times more likely to choose the option they believed the interviewer had made, and 5 times more likely with a foreign interviewer who needed a translator). Industry has turned the same points into rules in its guidance and industry codes. The two sides diverge after the interviews. Academia decides when to stop by whether knowledge has become sufficient, using saturation or information power (saturation at 9 to 17 interviews, Hennink and Kaiser 2022), whereas industry repeats rounds of 4 to 8 people and stops when the team is confident in its design decisions. For analysis, academia keeps verbatim records and systematic procedures, whereas industry groups findings on sticky notes while memories are fresh and allots one hour of analysis to every two hours of research. A study that compared anthropology and design students at the same site found that the techniques were the same and that the differences lay in the purpose of the inquiry and the treatment of participants' voices (Segelström 2013). In industry, interviewing has spread beyond researchers (in a practitioner survey, 61% named product designers as people who conduct research). Ethics is checked by institutional review in academia and by organizational guidance and industry codes in industry. At the business layer, academia has tested the customer discovery method itself with a randomized experiment (Camuffo et al. 2020). On generative AI, both sides agree that synthetic users should not replace people. For AI as the interviewer, quality comparable to that of human interviewers has been reported for structured interviewing, and many assessments hold that exploratory interviews require a person.

## Where does academic and industry guidance diverge?

**User interviewing** is the practice of meeting the people who use or buy something, before or while it is made, and listening to them to inform decisions about what to make.
Academia has discussed this practice as qualitative interviewing, as contextual inquiry (interviewing people within their context), and as generative research, in which participants are asked to make materials.
Industry calls it discovery interviews, customer interviews, or customer discovery.
The subject of the interviews ranges from how a screen is used to how a service works and to the hypotheses of a business.

This note draws on 32 academic works (peer-reviewed papers, a doctoral dissertation, a scholarly book, and 2 preprints) and 39 industry sources.
The industry sources consist of 14 guidance documents from public bodies and industry codes of research ethics; 12 materials from research firms, UX consultancies, design firms, and strategy consultancies; and 13 first-hand accounts by practitioners who have developed and practiced the methods.
Numbers, quotations, and claims were checked against the relevant passages of the full texts wherever these could be obtained.
When the full text of a work could not be obtained, "abstract" is added after the year, and the work is used only within the scope of its abstract (see Unverified items at the end).

Placed side by side, the conventions of asking are almost the same.
Both sides advise against asking people directly what they want.
Both advise asking about specific past events, not leading participants, listening in silence, and, where possible, listening in the person's own setting.
However, the guidance is written differently when it comes to how many people to interview before stopping, how to record and analyze what was heard, who interviews, and who checks the ethics.
Where does this divergence come from?

## Should interviewers ask about opinions or about events?

On the academic side, this convention of asking is supported by findings on the limits of self-report.
Nisbett and Wilson (1977) argued that when people report the causes of their own judgments, they answer by applying causal theories about whether a given kind of stimulus is likely to produce a given kind of response, instead of tracing their memory of the process.
Jerolmack and Khan (2014, abstract) state that what people say is a weak predictor of what they do and that interview and survey researchers often mistake self-reports for behavior.
They named this error the "attitudinal fallacy."
The same abstract argues that it is ethnography, not interviewing, that addresses the gap between attitudes and behavior.

On the design research side, Beyer and Holtzblatt (1995, opening paragraphs) wrote that conventional interviews and questionnaires are insufficient for designing today's applications and that designers need to understand people within the context of their own work.
As the title indicates, they proposed a relationship in which the designer apprentices with the customer.
Another method for collecting specific past events is the critical incident technique (CIT), whose 50-year history Butterfield et al. (2005, abstract) traced.
According to Gremler (2004, abstract), 141 studies in service research alone have used CIT.

Industry guidance sets out the same points as procedures.
The GOV.UK Service Manual asks interviewers to focus on stories and real examples during an interview and to avoid generalizations and talk about how things "should be."
Teresa Torres (2022) recommends starting with the question "Tell me about the last time you ...."
Ash Maurya (2017) writes that the only reliable evidence is what customers did in the past or are doing now, and that asking about the future, as in "Will you use ...," leads to bias.
Luo (2023) of Strategyzer recommends asking about facts and experiences over opinions and not asking whether people would buy or how much they would pay.
When a participant begins to speculate on "where else would you buy it?", Spiek and Moesta, who demonstrated the switch interview of Jobs to Be Done (JTBD), have that part removed from the record.
In the listening sessions of Indi Young (2015), the interviewer asks about everything that went through the person's mind during a recent event and asks no questions about solutions.

The two sides also agree on the prescription of reading what people say together with observation.
Whereas Jerolmack and Khan recommend ethnography, Nielsen Norman Group (NN/g; Rosala and Pernice 2023) lists two limitations of interviews: memory is unreliable, and the interviewer relies on self-report because observed behavior is not obtained.
IDEO.org's Design Kit writes that "What the person says is only one data point" and asks the interviewer to watch gestures and context.
For observation visits, GOV.UK sets out three levels: watching silently, asking occasional questions, and having the person explain each step as they go.
It describes asking occasional questions as the approach that balances naturalness and understanding.

## Does the interviewer shift the answers?

Answers are shifted by more than the wording of the questions.
Dell et al. (2012) conducted 450 interviews in Bangalore, India, and showed that participants were about 2.5 times more likely to choose the technology they believed the interviewer had made.
The two technologies were identical.
When the interviewer was a foreign researcher who required a translator, this bias increased to 5 times.
Even when the technology made by the interviewer was deliberately degraded, participants still chose it.

Industry codes address this bias through rules.
The Code of Conduct of the UK Market Research Society (2023) requires that data collection be designed so as not to lead participants toward a particular view.
On the GOV.UK user research blog, Spanton (2014) asks facilitators not to lead research toward the outcomes they anticipate, giving leading questions such as "How easy did you find this section of the site?" as examples.
Erika Hall (2013) writes that a good interview is a matter of keeping quiet and recommends using pauses to draw out the story.
Strategyzer and Steve Blank (2020) write that an interview is not a sales pitch.

Within this agreement, the depth of treatment differs.
Industry rules try to suppress the bias through wording and the interviewer's stance.
The results of Dell et al. show that the size of the bias changes with the social distance between the interviewer and the participant, and that the bias arises merely because participants think the interviewer made the item.
A designer who brings their own proposal to an interview is in a situation that overlaps with the condition in this experiment in which participants know that the interviewer made the item.
However, these figures were measured in a specific population in India, and the bias cannot be assumed to have the same size in every setting.

## How many people should be interviewed before stopping?

The way of deciding when to stop is a point at which the two sides clearly diverge.
Academia stops when knowledge has become sufficient.
Guest et al. (2006, abstract) analyzed interviews with 60 women in West Africa and reported that saturation occurred within the first 12 interviews and that the elements of the broad themes had appeared within 6 interviews.
Hennink and Kaiser (2022) systematically reviewed 23 studies that tested saturation empirically and found that interview studies reached saturation at 9 to 17 interviews, but stated that this range applied only to studies of relatively homogeneous populations with narrowly defined objectives.
Studies spanning multiple countries and studies that sought saturation in the meaning of codes required more interviews.
Malterud et al. (2016, abstract) criticized the inconsistent use of the term saturation across methodologies and proposed information power, the idea that the more information relevant to the study a sample holds, the fewer participants are needed.
In this proposal, the number depends on the aim of the study, the specificity of the sample, the use of established theory, the quality of dialogue, and the analysis strategy.

Industry uses the round as its unit and stops when the team is confident in its design decisions.
GOV.UK asks for a clear purpose for each round and states that a round of interviews or usability tests typically involves 4 to 8 people.
For the alpha phase, it writes that research is sufficient when the team is confident that its design solutions meet the needs of users, including people with disabilities.
In the 2023 practitioner survey of the company User Interviews (a self-selected sample of 929 people), the median number of participants in moderated studies was 8.
At the business layer, more people are interviewed.
Blank requires the students in his course to talk to 100 customers in 10 weeks (2020).

There are also efforts from the industry side to fill the space between these two criteria.
Rosala (2021) of NN/g writes that there is no fixed number, and states that applying the five-user guideline for usability testing to interviews is a mistake and that five is too few for most exploratory studies.
Rosala then recommends starting with a representative sample of 5 or 6 people and adding participants while analyzing until nothing new is learned, citing the saturation study of Guest et al. as the basis.
Nielsen and Landauer (1993, abstract), the source of the five-user guideline, presented a model that describes the number of usability problems found as a function of the number of test users, using a Poisson process, and the model does not address saturation in interviews.
Jared Spool (2011) recommends considering observation time over the number of people, and wrote that two hours with one person can be worth more than 15 minutes each with 8 people.
According to Spool, teams that do research only once a year struggle almost as much as teams that do no research, and once every six weeks appeared to be the minimum frequency for two hours of observation.
This observation is a rule of thumb that Spool drew from watching many teams, and it is not a tested causal relationship.

## How is what was heard handled after the interviews?

The differences grow larger at the stage at which what was heard is recorded, analyzed, and turned into outputs.
GOV.UK asks teams to analyze as soon as possible after each round, while memories are fresh, and sets one hour of analysis for every two hours of research as a guideline.
It holds that bringing together many people who observed the sessions reduces researcher bias, and it has teams group findings by affinity sorting, in which sticky notes are moved to form clusters.
The US digital.gov likewise states that an interview requires at least two people, an interviewer and a note-taker.
In academic qualitative research, researchers produce verbatim transcripts and identify themes through explicit procedures.
Braun and Clarke (2006, abstract) provided guidance for doing thematic analysis deliberately and rigorously, a method that was widely used in psychology but whose position had been unclear.

One study compared these two ways of handling material at the same site.
Segelström (2013) had two anthropology students and two teams of design students study the same site, an Advent market.
The anthropology students narrowed their focus early on the basis of the literature and kept their research question unchanged to the end.
They recorded and transcribed semi-structured interviews, and in the analysis they aimed to remain faithful to the voice of each individual.
The design students aimed for actionable insights and changed their initial focus whenever a more promising area appeared.
In the materials they presented, the voices of individual participants were not visible, and personas and user profiles were central.
Segelström concluded that both groups used the same techniques in similar ways and that the differences lay in the purpose of the inquiry and the treatment of participants' voices.
According to Segelström, these differences appeared most clearly in analysis and synthesis after the material had been gathered.
This comparison is between students and comes from a single site.
Although it does not compare academia and industry as such, it shows that the difference between interviewing to know and interviewing to make appears in the handling after the interviews and not in the techniques.

On the academic side, too, there are arguments that treat this difference as a problem.
Dourish (2006) criticized the practice in HCI of evaluating ethnographic studies by their "implications for design."
Dourish argued that a fixation on drawing out implications misreads the nature of ethnography and that the value of ethnography lies in the models and ways of thinking that it provides.
The "implications for design" that Dourish sought to remove from the yardstick for evaluating ethnography were, for the design students, the very purpose of the inquiry.

At the same time, academia has also studied ways to interview quickly.
Millen (2000, abstract) proposed rapid ethnography to fit the short cycles of product development and listed its elements: narrowing the focus and scope, using key informants, recording with multiple observers, and analyzing collaboratively.
Vindrola-Padros and Johnson (2020) reviewed techniques of rapid qualitative research and reported that the ways of speeding up conventional methods fall into two groups: not producing verbatim transcripts (by collecting and analyzing data simultaneously or by coding directly from audio or video) and speeding up transcription.
The same review names two drawbacks: relying on notes alone loses detail, and obtaining the same quality as conventional methods requires experienced researchers.
The industry practice of grouping findings on sticky notes belongs to the former type, and the academic review names the costs of that type.

## Who conducts the interviews?

The two sides also differ in their assumptions about who the interviewer is.
In industry, interviewing by people who are not researchers is spreading.
In the 2023 survey of User Interviews, 26% of respondents were people who do research without being researchers, such as designers, PMs, marketers, and data analysts.
In organizations without ResearchOps, non-researchers made up 61% of the people conducting research.
In the 2024 survey of Maze (1,234 people, distributed by Maze itself), product designers were named as people who conduct research by 61%, PMs by 38%, and marketers by 17%.
In the tabulation by MeasuringU (2025) of the 2024 UXPA survey (444 people in 37 countries), 75% used "user research/interviews/surveys," the most common of the UX methods.
In the 2025 survey of User Interviews, interviews were the top method supported by ResearchOps, at 92%.

Practitioners themselves have concerns about this spread.
In the 2023 survey of User Interviews, sentiment toward research democratization (research carried out by people other than researchers) averaged 2.95 out of 5, and 45% of 526 free-text responses were negative.
The most common concern was a decline in quality.
Awamura (2023) of freee recalled that interviews by PMs and designers had come to include some that did not follow the basics, such as interviews with many closed questions.
According to Awamura, maintaining the kit that compiled the procedures could not keep pace, and arranging appointments, incentives, and confidentiality could no longer be handled in parallel with fieldwork, so the company hired a dedicated ResearchOps specialist.
Pernice (2022) of NN/g recommends that experienced researchers handle complex studies and that easier studies be delegated with coaching and resources.
Kusano (2025) of Merpay uses a mechanism called Weekly UXR, which holds research sessions regularly every week, to lower the cost of designing research from scratch and to make it easy for internal stakeholders to start using research.

Academia treats the interviewer's skill as part of the validity of the method.
One of Segelström's (2013) motives for comparing anthropology and design was a criticism, widespread in the academic literature, that designers lack sufficient skill and preparation for field research.
There is also research on the gap between research and practice.
When Colusso et al. (2017) interviewed 22 practitioners, they found that academic research was rarely used in practice and that practitioners preferred actionable guidance.
Gray et al. (2014, abstract) asked interaction designers how they use affinity diagramming and affordances, and presented a model of the paths by which ideas from practice bubble up to research and theory trickles down to practice.
In the collection for this note, no academic study was found that measured the quality of interviews conducted by people who are not researchers.

## Who checks the ethics?

The content of research ethics is quite similar on the two sides.
GOV.UK requires informed consent from all participants and asks researchers to tell participants that participation is voluntary and can be stopped at any time, what the data will be used for, and how long it will be kept.
For children and vulnerable people, consent is obtained from both the person and a parent, guardian, or similar.
The ICC/ESOMAR International Code (2025) states that anyone has the right to refuse to participate and that the information collected is limited to what is directly relevant to the purpose of the research.

What differs is who does the checking.
Some of the academic interview studies read for this note explicitly state approval by an institutional ethics review board (IRB) (Kapania et al. 2025).
The GOV.UK page on consent sets out how to obtain consent and handle data, but it does not assume review by a third party.
At the Digital Agency's study group on revising its service design guidelines (2026), committee members expressed the opinion that the guidelines should add that ethics committee approval may be required, and that work involving heavy burdens or sensitive information should be referred to an ethics committee.
This is an opinion of committee members, not a decision.
This discussion indicates that in industry and government, ethics checks are handled by organizational guidance and industry codes, and that third-party review is treated as an exception.

Academia has also discussed whom to bring before the interviewer as a question of power.
Harrington et al. (2019, abstract) argued that participatory design workshops tend to become activities of the affluent and privileged, and that there are tensions involving the historical context of research, access to communities, the reception of materials, and the harm of collecting personal narratives.
Costanza-Chock (2020, abstract) argued that design principles that claim universality erase people who are intersectionally disadvantaged, and proposed design led by marginalized communities.
Industry and government also seek diverse participants.
GOV.UK asks teams to include participants with disabilities even when the service currently has no users with disabilities, giving as the reason that barriers should not be created for people who may want to use the service in the future.
The DS-670.1 of Japan's Digital Agency (2026) includes, within the identification of diverse users, the idea of inclusive design, in which people with disabilities and older people take part at each stage of a project and contribute to design and evaluation.
Although they reach the same conclusion of seeking diverse participants, Harrington et al. and Costanza-Chock ground it in structural inequality, whereas GOV.UK grounds it in not creating barriers for future users.

## Does interviewing change across UIs, services, and businesses?

At the UI layer, interviewing sits next to the evaluation of prototypes.
GOV.UK asks teams to conduct discovery research, which learns about users and their problems and needs, before planning, designing, and building a service.
In the following alpha phase, teams test prototypes with intended users and continue the research until they are confident that the design solutions meet those needs.
This is where the bias found by Dell et al. is most likely to take effect.
When the person who made a prototype places it in front of participants, the conditions are in place for participants to choose what the interviewer made.

At the service layer, the number of people to talk to increases.
The Double Diamond of the UK Design Council treats the first phase, Discover, as a phase of understanding the problem instead of assuming it, and includes talking to and spending time with the people affected by the issue.
Segelström (2013) positioned service design as a "stakeholder-centric" field that involves users, staff, and other stakeholders in research.
In Segelström's participant observation at three service design agencies, the agencies differed in whether they treated stakeholders as sources of inspiration or as co-creators of ideas, and these differences were far larger than the agencies' outward descriptions suggested.
In Segelström's interviews, practitioners gave three reasons for visualizing research insights: to articulate the insights, to communicate them to clients, and to keep the material alive.
Trischler et al. (2018) reported from an idea contest at a library that ideas from teams that included customers scored higher on user benefit and novelty, and lower on feasibility, than ideas from teams of in-house professionals only.
This result held only for teams that were cohesive and developed ideas collaboratively.
This result concerns the effect of including customers in design, not the effect of interviewing them.
The contextmapping of Sleeswijk Visser et al. (2005, abstract) is a generative research technique developed from research projects and many years of industrial practice, and it is an example of a method born at the boundary between academia and industry.

At the business layer, interviewing becomes hypothesis testing.
Blank's 2026 course requires at least 10 recorded interviews with real people, half of them in person, and has students write up each interview in the format of "hypothesis/experiment/expected result/actual result/path forward."
Maurya writes that when searching for early adopters, the key question is "when," not "who," and recommends looking at behavioral triggers over attributes.

Academia has tested the method itself at this layer.
Camuffo et al. (2020, abstract) randomly assigned 116 Italian startups and reported that the group taught to formulate hypotheses and test them rigorously performed better, was more likely to pivot to a different idea, and did not drop out more often.
Leatherbee and Katila (2020) followed 152 teams in the NSF I-Corps program, each of which conducted close to 100 interviews in 8 weeks.
The elements of hypothesis formulation, testing, and convergence on a business idea were connected as predicted, and team members trained in MBA programs resisted the method but recognized its value afterward.
I-Corps is an NSF program that Blank created with the lean startup method, and its figure of close to 100 interviews also matches the 100 people that Blank requires in his course.
Shepherd and Gruber (2021, abstract) divided lean startup into five components, including customer development, and sought to narrow the gap between academia and practice by supplementing these components with research findings.
In the collection for this note, there was no study that tested the effects of interviewing methods at the UI and service layers with randomized experiments.

## Does generative AI enter as the interviewer or as the interviewee?

Generative AI is entering two positions in user interviewing.
One is the interviewer.
In a field study with about 600 people, Xiao et al. (2020) compared conversational surveys conducted by a chatbot, which interprets open-ended answers and asks follow-up questions when needed, with a typical online questionnaire.
With the chatbot, participant engagement was higher, and response quality was higher in informativeness, relevance, specificity, and clarity.
In a small study that randomly assigned university students to AI and human interviewers, Wuttke et al. (2025) reported that data quality was comparable to that of conventional methods.
Deviations from the protocol were similarly frequent for both, but their content differed: human interviewers tended to neglect the step of restating answers to confirm them.
Wuttke et al. (2026, preprint) report, with 571 people, that AI-conducted interviews revealed differences in views that standardized questionnaires could not capture, and that people who completed the interviews rated them as equal to or better than questionnaires.
However, the proportion of people who completed the interviews differed across conditions.

Assessments from industry trials are somewhat more critical.
Rosala (2026) of NN/g tried two AI interviewing tools with 10 research leaders and ResearchOps professionals.
Three of them said that the conversation felt natural and five said that they felt at ease, and both tools showed the same problems: interrupting, staying silent for a long time after answers, and repeating the same question.
Rosala concludes that AI interviews can reach people quickly and widely but cannot replace the in-depth semi-structured interviews that people conduct.

The other position is the interviewee, that is, the participant.
Here the conclusions of the two sides are closest.
Agnew et al. (2024) collected proposals to replace human participants with large language models and argued that the motivations for these proposals lie in reducing costs and diversifying the data collected.
They also argued that the proposals conflict with the basic values of research with people: representation, inclusion, and understanding.
In a study with 4 models and 3,200 people across 16 demographic groups, Wang et al. (2025) showed that the models misportray and flatten groups.
When Kapania et al. (2025) interviewed 19 qualitative researchers, the researchers were at first surprised that the models returned narratives similar to those of human participants, but as the interaction went on, they named limitations such as taking away consent and agency and lacking texture and contextual depth.
Hämäläinen et al. (2023, abstract) showed that GPT-3 can produce plausible accounts of experiencing games as art, while writing that synthetic data can be used for ideation and pilot studies but that findings must always be verified with real data.
Park et al. (2026, preprint) built agents that simulate individuals from self-reports such as interviews of about two hours, and the records of interviews themselves are beginning to serve as material for synthetic participants.

Industry practitioners also do not use synthetic users as substitutes for people.
When Rosala and Moran (2024) of NN/g reproduced past studies with synthetic users, the synthetic users said that they had completed every course they had taken and showed interest in everything.
The two authors hold that synthetic users can be used for desk research and for generating hypotheses but not for final decisions, and they ask that synthetic findings not be presented as findings from real users.
Torres (2024) wrote that generative AI should not be used to replace discovery with real humans.
Steve Portigal (2023) introduced Niloufar Salehi's report on trying a synthetic user service and encouraged readers to share it when synthetic users are recommended in place of real research.
Blank (2026) allows synthetic interviews with AI customer personas in his course, but writes that the live interviews grade the synthetic ones, not the other way around.
The exception is the replication of quantitative studies: Bain & Company (2026) reports that digital twins built from past respondent-level data reproduced about 90% of the key results of a past conjoint study.
This 90% is a claim by one company about one study, and the article does not state what was counted as a key result or what was judged to be a replication.
In the 2025 survey of User Interviews, 80% used AI in research, and 91% worried about output accuracy and hallucinations.

## Shared ground and differences by topic

| Topic | Shared ground | Academic emphasis | Industry emphasis |
|---|---|---|---|
| What to ask about | Ask about specific past events instead of opinions or predictions of the future. Read what people say together with observation | Theories of the limits of self-report (Nisbett and Wilson, Jerolmack and Khan), CIT, contextual inquiry | Procedural rules (GOV.UK, Torres, Maurya, Strategyzer, switch interview) |
| Leading and participant response bias | Do not lead. Listen in silence. Do not sell | Measuring the size of the bias (about 2.5 times in Dell et al., 5 times with a foreign interviewer) | Rules on wording and stance (MRS Code of Conduct, GOV.UK, Hall) |
| When to stop | No fixed number. Add participants while analyzing | Sufficiency of knowledge (saturation, information power; the guide of 9 to 17 interviews is limited to homogeneous populations and narrow objectives) | Repeat rounds of 4 to 8 people and stop when confident in design decisions. Think in terms of observation time and frequency |
| Recording and analysis | Combine observations into themes or findings | Verbatim transcripts and explicit procedures. Also studies fast methods and names their costs | Group findings on sticky notes while memories are fresh. One hour of analysis for every two hours of research |
| Outputs | Understanding of participants' worlds | Descriptions that keep participants' voices, models, ways of thinking | Actionable insights, personas, design opportunities |
| Who interviews | The interviewer's skill affects the results | Assumes trained researchers | Extends interviewing to non-researchers and supports quality through training, kits, and ResearchOps |
| Ethics | Consent, voluntary participation, the right to withdraw, minimizing the information collected, diverse participants | Institutional ethics review. Critiques of power and inequality (Harrington et al., Costanza-Chock) | Organizational guidance and industry codes. Third-party review is the exception. Do not create people who cannot access the service |
| Business layer | Test hypotheses with facts about behavior, without asking whether people would buy | Tests the method itself with randomized experiments and longitudinal tracking | Procedures in courses and practice (Blank, Maurya, Strategyzer) |
| Generative AI | Do not use synthetic users as substitutes for people | AI interviewers comparable to humans in structured interviewing. Measuring the harms of synthetic participants | AI interviewers unsuited to exploratory interviews. Synthetic data limited to desk research and hypothesis generation |

## Where do the differences come from?

The differences in the table can be read as differences in what the findings from interviews must justify, and to whom.
This reading is this note's own interpretation.
Academic interviewing needs to let readers who were not present verify its knowledge claims.
Therefore, it explains its stopping point by appeal to saturation, keeps verbatim records and procedures, preserves participants' voices in its descriptions, and has a third party check its ethics.
Industry interviewing needs to support, right away, the same team's decision about what to build next.
Therefore, it repeats rounds, stops when the team is confident, groups findings while memories are fresh, and brings many people into the interviews.
In Segelström's comparison, the differences between groups that used the same techniques appeared most clearly in analysis and synthesis, which is consistent with this reading.

This boundary is moving.
On the industry side, NN/g cites saturation studies to explain numbers of participants.
Members of the Digital Agency's study group have proposed writing the following into the guidelines: an indicative number of participants for qualitative research that includes minorities, the use of multiple analysts, triangulation, and consultation with an ethics committee.
On the academic side, studies of rapid ethnography and rapid qualitative research have measured what is lost in exchange for speed.
At the business layer, academia is testing lean startup, which emerged from practice, with randomized experiments.

## What this investigation can and cannot say

What can be said is that academia and industry largely agree on the conventions of asking, and that the differences cluster in how the material is handled after the interviews: when to stop, recording and analysis, the interviewer, the checking of ethics, and the form of outputs.
These differences are not a matter of one side being right.
Each side makes choices that fit what the findings from the interviews will be used for.

Many things cannot be said.
The industry figures include self-selected surveys that are biased toward each company's own customers (User Interviews, Maze) and a claim whose methods are not published (Bain & Company), and the surveys cannot be compared with one another.
The academic findings on numbers of participants (saturation and information power) come from interview studies in health and the social sciences, and in the collection for this note, no study was found that empirically established a criterion for the number of participants in design interviewing.
Segelström's comparison is a result from one site with students, and the measurements by Dell et al. are results from a specific population in India.
Primary sources in which service design practitioners describe their own interviewing procedures could not be secured (the relevant chapter of This is Service Design Doing could not be obtained).
No experiment that tested the effects of interviewing methods at the UI and service layers was found, and no study that measured the quality of interviews by people who are not researchers was found either.

## Related notes

- [Qualitative and Quantitative Research: In the Context of Design](https://jamjet-iqbal.com/en/notes/qualitative-quantitative-design-research/): Differences between qualitative and quantitative research, and how to choose methods in design research
- [Is the Primacy of Qualitative Methods in Design Research Academically Mainstream?](https://jamjet-iqbal.com/en/notes/design-research-methodology-mainstream/): The position of qualitative methods in design research
- [A Genealogy of Practitioner Methods for Reframing Problems: Who Made Them, Traced to Their Origins](https://jamjet-iqbal.com/en/notes/problem-definition-methods-industry/): Practical methods for reframing problems, including the three strands of Jobs-to-be-Done
- [Which standard can design and new-business work actually sit on?](https://jamjet-iqbal.com/en/notes/design-work-process-standards/): ISO 9241-210 (human-centered design) and process standards
- [AI in the Design Industry: An Academic Review (2026)](https://jamjet-iqbal.com/en/notes/ai-in-design-literature/): An academic review of AI use in design, including synthetic users

## Unverified items

No unverified statement remains in the main claims.
The full texts of the following works could not be obtained (many publishers and institutional repositories refused automated retrieval).
These works were therefore used only within the scope of their abstracts: Beyer and Holtzblatt (1995, up to the opening paragraphs), Braun and Clarke (2006), Butterfield et al. (2005), Camuffo et al. (2020), Costanza-Chock (2020), Gray et al. (2014), Gremler (2004), Guest et al. (2006), Hämäläinen et al. (2023), Harrington et al. (2019), Jerolmack and Khan (2014), Malterud et al. (2016), Millen (2000), Nielsen and Landauer (1993), Shepherd and Gruber (2021), and Sleeswijk Visser et al. (2005).
Of these, for Camuffo et al., Harrington et al., and Jerolmack and Khan, the full text was found only in copies whose permission from the source could not be confirmed or on pages that refuse automated retrieval, and these were not used.
Wuttke et al. (2026) and Park et al. (2026) are preprints, and peer-reviewed versions could not be confirmed.

## References

All sources accessed 2026-11-01. The ledgers and verification records are `papers.md` and `industry.md` in `source/review/design-user-interview-practice/`.

### Academic

- Agnew, William, A. Stevie Bergman, Jennifer Chien, Mark Díaz, Seliem El-Sayed, Jaylen Pittman, Shakir Mohamed, and Kevin R. McKee. 2024. "The Illusion of Artificial Inclusion." *Proceedings of the CHI Conference on Human Factors in Computing Systems*: 1–12. https://doi.org/10.1145/3613904.3642703
- Beyer, Hugh R., and Karen Holtzblatt. 1995. "Apprenticing with the Customer." *Communications of the ACM* 38 (5): 45–52. https://doi.org/10.1145/203356.203365
- Braun, Virginia, and Victoria Clarke. 2006. "Using Thematic Analysis in Psychology." *Qualitative Research in Psychology* 3 (2): 77–101. https://doi.org/10.1191/1478088706qp063oa
- Butterfield, Lee D., William A. Borgen, Norman E. Amundson, and Asa-Sophia T. Maglio. 2005. "Fifty Years of the Critical Incident Technique: 1954–2004 and Beyond." *Qualitative Research* 5 (4): 475–497. https://doi.org/10.1177/1468794105056924
- Camuffo, Arnaldo, Alessandro Cordova, Alfonso Gambardella, and Chiara Spina. 2020. "A Scientific Approach to Entrepreneurial Decision Making: Evidence from a Randomized Control Trial." *Management Science* 66 (2): 564–586. https://doi.org/10.1287/mnsc.2018.3249
- Colusso, Lucas, Cynthia L. Bennett, Gary Hsieh, and Sean A. Munson. 2017. "Translational Resources: Reducing the Gap Between Academic Research and HCI Practice." *Proceedings of DIS 2017*: 957–968. https://doi.org/10.1145/3064663.3064667
- Costanza-Chock, Sasha. 2020. *Design Justice: Community-Led Practices to Build the Worlds We Need*. MIT Press. https://doi.org/10.7551/mitpress/12255.001.0001
- Dell, Nicola, Vidya Vaidyanathan, Indrani Medhi, Edward Cutrell, and William Thies. 2012. "'Yours Is Better!' Participant Response Bias in HCI." *Proceedings of CHI 2012*: 1321–1330. https://doi.org/10.1145/2207676.2208589
- Dourish, Paul. 2006. "Implications for Design." *Proceedings of CHI 2006*: 541–550. https://doi.org/10.1145/1124772.1124855
- Gray, Colin M., Erik Stolterman, and Martin A. Siegel. 2014. "Reprioritizing the Relationship Between HCI Research and Practice: Bubble-Up and Trickle-Down Effects." *Proceedings of DIS 2014*: 725–734. https://doi.org/10.1145/2598510.2598595
- Gremler, Dwayne D. 2004. "The Critical Incident Technique in Service Research." *Journal of Service Research* 7 (1): 65–89. https://doi.org/10.1177/1094670504266138
- Guest, Greg, Arwen Bunce, and Laura Johnson. 2006. "How Many Interviews Are Enough? An Experiment with Data Saturation and Variability." *Field Methods* 18 (1): 59–82. https://doi.org/10.1177/1525822X05279903
- Hämäläinen, Perttu, Mikke Tavast, and Anton Kunnari. 2023. "Evaluating Large Language Models in Generating Synthetic HCI Research Data: A Case Study." *Proceedings of CHI 2023*: 1–19. https://doi.org/10.1145/3544548.3580688
- Harrington, Christina, Sheena Erete, and Anne Marie Piper. 2019. "Deconstructing Community-Based Collaborative Design: Towards More Equitable Participatory Design Engagements." *Proceedings of the ACM on Human-Computer Interaction* 3 (CSCW): 1–25. https://doi.org/10.1145/3359318
- Hennink, Monique, and Bonnie N. Kaiser. 2022. "Sample Sizes for Saturation in Qualitative Research: A Systematic Review of Empirical Tests." *Social Science & Medicine* 292: 114523. https://doi.org/10.1016/j.socscimed.2021.114523
- Jerolmack, Colin, and Shamus Khan. 2014. "Talk Is Cheap: Ethnography and the Attitudinal Fallacy." *Sociological Methods & Research* 43 (2): 178–209. https://doi.org/10.1177/0049124114523396
- Kapania, Shivani, William Agnew, Motahhare Eslami, Hoda Heidari, and Sarah E. Fox. 2025. "'Simulacrum of Stories': Examining Large Language Models as Qualitative Research Participants." *Proceedings of CHI 2025*: 1–17. https://doi.org/10.1145/3706598.3713220
- Leatherbee, Michael, and Riitta Katila. 2020. "The Lean Startup Method: Early-Stage Teams and Hypothesis-Based Probing of Business Ideas." *Strategic Entrepreneurship Journal* 14 (4): 570–593. https://doi.org/10.1002/sej.1373
- Malterud, Kirsti, Volkert Dirk Siersma, and Ann Dorrit Guassora. 2016. "Sample Size in Qualitative Interview Studies: Guided by Information Power." *Qualitative Health Research* 26 (13): 1753–1760. https://doi.org/10.1177/1049732315617444
- Millen, David R. 2000. "Rapid Ethnography: Time Deepening Strategies for HCI Field Research." *Proceedings of DIS 2000*: 280–286. https://doi.org/10.1145/347642.347763
- Nielsen, Jakob, and Thomas K. Landauer. 1993. "A Mathematical Model of the Finding of Usability Problems." *Proceedings of INTERCHI '93*: 206–213. https://doi.org/10.1145/169059.169166
- Nisbett, Richard E., and Timothy D. Wilson. 1977. "Telling More Than We Can Know: Verbal Reports on Mental Processes." *Psychological Review* 84 (3): 231–259. https://doi.org/10.1037/0033-295X.84.3.231
- Park, Joon Sung, Carolyn Q. Zou, Jonne Kamphorst, Niles Egan, Aaron Shaw, Benjamin Mako Hill, Carrie Cai, Meredith Ringel Morris, Percy Liang, Robb Willer, and Michael S. Bernstein. 2026. "LLM Agents Grounded in Self-Reports Enable General-Purpose Simulation of Individuals." arXiv:2411.10109 (preprint, v3). https://arxiv.org/abs/2411.10109
- Segelström, Fabian. 2013. *Stakeholder Engagement for Service Design: How Service Designers Identify and Communicate Insights*. Linköping Studies in Arts and Science 586, Linköping University (doctoral dissertation). https://doi.org/10.3384/diss.diva-97320
- Shepherd, Dean A., and Marc Gruber. 2021. "The Lean Startup Framework: Closing the Academic–Practitioner Divide." *Entrepreneurship Theory and Practice* 45 (5): 967–998. https://doi.org/10.1177/1042258719899415
- Sleeswijk Visser, Froukje, Pieter Jan Stappers, Remko van der Lugt, and Elizabeth B.-N. Sanders. 2005. "Contextmapping: Experiences from Practice." *CoDesign* 1 (2): 119–149. https://doi.org/10.1080/15710880500135987
- Trischler, Jakob, Simon J. Pervan, Stephen J. Kelly, and Don R. Scott. 2018. "The Value of Codesign: The Effect of Customer Involvement in Service Design Teams." *Journal of Service Research* 21 (1): 75–100. https://doi.org/10.1177/1094670517714060
- Vindrola-Padros, Cecilia, and Ginger A. Johnson. 2020. "Rapid Techniques in Qualitative Research: A Critical Review of the Literature." *Qualitative Health Research* 30 (10): 1596–1604. https://doi.org/10.1177/1049732320921835
- Wang, Angelina, Jamie Morgenstern, and John P. Dickerson. 2025. "Large Language Models That Replace Human Participants Can Harmfully Misportray and Flatten Identity Groups." *Nature Machine Intelligence* 7 (3): 400–411. https://doi.org/10.1038/s42256-025-00986-z
- Wuttke, Alexander, Matthias Aßenmacher, Christopher Klamm, Max M. Lang, Quirin Würschinger, and Frauke Kreuter. 2025. "AI Conversational Interviewing: Transforming Surveys with LLMs as Adaptive Interviewers." *Proceedings of LaTeCH-CLfL 2025*: 179–204. https://doi.org/10.18653/v1/2025.latechclfl-1.17
- Wuttke, Alexander, Max M. Lang, Christopher Klamm, Quirin Würschinger, and Frauke Kreuter. 2026. "AI Conversational Interviewing: Scaling Up Semi-Structured and In-Depth Interviews." arXiv:2606.20064 (preprint). https://arxiv.org/abs/2606.20064
- Xiao, Ziang, Michelle X. Zhou, Q. Vera Liao, Gloria Mark, Changyan Chi, Wenxi Chen, and Huahai Yang. 2020. "Tell Me About Yourself: Using an AI-Powered Chatbot to Conduct Conversational Surveys with Open-Ended Questions." *ACM Transactions on Computer-Human Interaction* 27 (3): 1–37. https://doi.org/10.1145/3381804

### Public bodies, standards, and codes of research ethics

- Design Council. n.d. "The Double Diamond." https://www.designcouncil.org.uk/our-resources/the-double-diamond/
- Digital Agency. 2026. "DS-670.1 Usability Guidelines" [DS-670.1 ユーザビリティガイドライン] (decided by the board of secretaries of the Digital Society Promotion Council, 2026-06-12). https://www.digital.go.jp/assets/contents/node/basic_page/field_ref_resources/e2a06143-ed29-4f1d-9c31-0f06fca67afc/a0942cf4/20260612_usability_guidelines.pdf
- Digital Agency. 2026. "The 4th Meeting of the Study Group on the Revision of Service Design Guidelines" [第 4 回サービスデザイン関連ガイドライン改訂に係る検討会]. https://www.digital.go.jp/councils/service-design-related-guidelines-revision/99fd3421-0231-4c10-a935-992c0eeef48f
- digital.gov (U.S. General Services Administration). n.d. "Interviews and Observations." HCD Guide. https://digital.gov/guides/hcd/discovery-operations/interviews
- GOV.UK Service Manual. 2016. "User Research in Discovery." https://www.gov.uk/service-manual/user-research/user-research-in-discovery
- GOV.UK Service Manual. 2017. "User Research in Alpha." https://www.gov.uk/service-manual/user-research/user-research-in-alpha
- GOV.UK Service Manual. 2017. "Using In-Depth Interviews." https://www.gov.uk/service-manual/user-research/using-in-depth-interviews
- GOV.UK Service Manual. 2017. "Contextual Research and Observation." https://www.gov.uk/service-manual/user-research/contextual-research-and-observation
- GOV.UK Service Manual. 2016. "Plan a Round of User Research." https://www.gov.uk/service-manual/user-research/plan-round-of-user-research
- GOV.UK Service Manual. 2026. "Finding Participants for User Research." (updated 2026-09-22) https://www.gov.uk/service-manual/user-research/find-user-research-participants
- GOV.UK Service Manual. 2016. "Analyse a Research Session." https://www.gov.uk/service-manual/user-research/analyse-a-research-session
- GOV.UK Service Manual. 2018. "Getting Informed Consent for User Research." https://www.gov.uk/service-manual/user-research/getting-users-consent-for-research
- ICC and ESOMAR. 2025. *ICC/ESOMAR International Code on Market, Opinion and Social Research and Data Analytics*. https://community.esomar.org/uploads/public/knowledge-and-standards/codes-and-guidelines/ICCESOMAR-International-Code_English.pdf
- Market Research Society. 2023. *MRS Code of Conduct*. https://www.mrs.org.uk/pdf/MRS-code-of-conduct-2023.pdf

### Research firms, consultancies, and design firms

- Bain & Company. 2026. "Synthetic Customers Earn Their Stripes." https://www.bain.com/insights/synthetic-customers-earn-their-stripes/
- IDEO.org. n.d. "Interview." Design Kit. https://www.designkit.org/methods/interview.html
- Luo, Lucy. 2023. "8 Tips for Conducting Interviews That Deliver Relevant Customer Insights." Strategyzer. https://www.strategyzer.com/library/8-tips-for-conducting-interviews-that-deliver-relevant-customer-insights
- Maze. 2024. *The Future of User Research Report 2024*. https://maze.co/resources/user-research-report-2024/
- MeasuringU. 2025. "The Methods UX Professionals Use (2024)." https://measuringu.com/ux-methods-2024/
- Pernice, Kara. 2022. "Democratize User Research in 5 Steps." Nielsen Norman Group. https://www.nngroup.com/articles/democratize-user-research/
- Rosala, Maria. 2021. "How Many Participants for a UX Interview?" Nielsen Norman Group. https://www.nngroup.com/articles/interview-sample-size/
- Rosala, Maria. 2026. "AI-Moderated Interviews: If, When, and How to Use Them." Nielsen Norman Group. https://www.nngroup.com/articles/ai-interviewers/
- Rosala, Maria, and Kate Moran. 2024. "Synthetic Users: If, When, and How to Use AI-Generated 'Research'." Nielsen Norman Group. https://www.nngroup.com/articles/synthetic-users/
- Rosala, Maria, and Kara Pernice. 2023. "User Interviews 101." Nielsen Norman Group. https://www.nngroup.com/articles/user-interviews/
- User Interviews. 2023. *State of User Research 2023 Report*. https://www.userinterviews.com/state-of-user-research-2023-report
- User Interviews. 2025. *The State of User Research Report 2025*. https://www.userinterviews.com/state-of-user-research-report

### First-hand accounts by practitioners

- Awamura, Norihisa. 2023. "Design Research at freee: Past and Future" [freee のデザインリサーチのこれまでとこれから]. Presentation slides, Research Conference 2023. https://speakerdeck.com/researchconf/research-conference-2023-freeenodezainrisatinokoremadetokorekara
- Blank, Steve. 2020. "Customer Discovery in the Time of the Covid-19 Virus." https://steveblank.com/2020/04/07/customer-discovery-in-the-time-of-the-covid-19-virus/
- Blank, Steve. 2026. "Lean LaunchPad: The Next Generation." https://steveblank.com/2026/09/30/lean-launchpad-the-next-generation/
- Hall, Erika. 2013. "Interviewing Humans." *A List Apart*. https://alistapart.com/article/interviewing-humans/
- Kusano, Koki. 2025. "A Study of UX Research Practice Cases in Merpay's Service Design" [メルペイのサービスデザインにおける UX リサーチ実践事例の考察]. Serviceology Web Magazine. https://magazine.serviceology.org/2025/06/20/2503/
- Maurya, Ash. 2017. "Find Better Problems Worth Solving with the Customer Forces Canvas." https://ashmaurya.com/blog/find-better-problems-worth-solving-with-the-customer-forces-canvas
- Portigal, Steve. 2023. "Superficial, Stereotyped User Research Bullshit." https://portigal.com/superficial-stereotyped-user-research-bullshit/
- Spanton, Jay. 2014. "What Makes a Good User Research Facilitator?" User Research in Government (GOV.UK blog). https://userresearch.blog.gov.uk/2014/07/11/what-makes-a-good-user-research-facilitator/
- Spiek, Chris, and Bob Moesta. n.d. "The Mattress Interview: A JTBD Switch Interview (Full Transcript)." Jobs to Be Done Radio. https://jobstobedone.org/radio/the-mattress-interview-part-one/
- Spool, Jared M. 2011. "Fast Path to a Great UX: Increased Exposure Hours." Center Centre. https://articles.centercentre.com/user_exposure_hours/
- Torres, Teresa. 2022. "Customer Interviews: How to Recruit, What to Ask, and How to Synthesize What You Learn." (updated 2026-09-29) Product Talk. https://www.producttalk.org/customer-interviews/
- Torres, Teresa. 2024. "Don't Use Generative AI to Replace Discovery with Real Humans." Product Talk. https://www.producttalk.org/2024/06/generative-ai-discovery/
- Young, Indi. 2015. "A New Way to Listen." *A List Apart*. https://alistapart.com/article/a-new-way-to-listen