Interviewing Subject Matter Experts for Technical Content

A product engineer gives a 45-minute interview about a new sensor. The transcript is accurate and almost useless: general statements about “robust design,” a list of features, and none of the judgment calls that make the product work. The problem isn’t the engineer. Interviewing subject matter experts is a skill, and the default interview, “tell me about the product,” gets the default answer. Research on knowledge elicitation explains why experts struggle to describe what they know, and offers methods that get past it. This guide turns those methods into steps you can use in your next technical interview.

Key Takeaways

• People are often poor at reporting the reasons behind their own judgments; Nisbett and Wilson’s classic review argued that such reports often reflect plausible theories rather than direct access to the process.
• Experts underestimate what novices don’t know: in Hinds’s 1999 study, experts were worse than intermediates at predicting how long novices would take on a task.
• The critical decision method (Klein, Calderwood and MacGregor, 1989) structures interviews around a specific, non-routine case and goes through it in several passes, probing for cues, goals and options at each decision point.
• The cognitive interview, developed for eyewitness interviewing, increased the amount of correct information recalled compared with standard interviews in Memon and colleagues’ 2010 meta-analysis, with a small increase in errors, which is why claims still need checking.
• Practical rule: ask about a real case, sweep it more than once, ask what a novice would miss, have the expert show or sketch, and send every claim back for review.

The problem: the default interview gets the brochure

“Tell me about the product” invites a summary the expert has given many times before, usually the brochure version. The details that make content useful to an engineering reader, such as why a particular filter was chosen, what goes wrong in the field and what the expert checks first, rarely come out unless the interviewer asks for them in a specific way.

Experts also compress. Steps that took years to learn feel obvious to them, and they leave them out without noticing. A writer who doesn’t know the subject well can’t tell what’s missing.

What the research says

Verbal reports are incomplete

Nisbett and Wilson’s “Telling more than we can know: Verbal reports on mental processes” (Psychological Review, 1977) reviewed evidence that people often can’t accurately report the processes behind their judgments, and that their explanations can reflect plausible theories about what should have influenced them. For interviewing, the implication is to ask about what the expert did and noticed in specific situations, not to ask them to theorize about how they think in general.

Experts lose sight of the novice’s view

Hinds’s “The curse of expertise” (Journal of Experimental Psychology: Applied, 1999) found that people with the most expertise were worse at predicting novice performance than people with intermediate experience. In an interview, this shows up as skipped steps and undefined terms. Asking directly “what would someone new to this get wrong?” helps, and so does interviewing a second, less senior person.

Structured, case-based methods get more

Hoffman, Shadbolt, Burton and Klein’s “Eliciting knowledge from experts: A methodological analysis” (Organizational Behavior and Human Decision Processes, 1995) compared knowledge-elicitation methods and discussed their relative efficiency, noting that unstructured interviews are not always the most productive way to capture expert knowledge. Klein, Calderwood and MacGregor’s “Critical decision method for eliciting knowledge” (IEEE Transactions on Systems, Man, and Cybernetics, 1989) describes a retrospective interview built around one specific, non-routine incident. The interviewer goes through the incident several times: first the overall story, then a timeline, then the decision points, probing at each for the cues the expert noticed, the goals they had, the options they considered and what they expected to happen.

Cognitive interviewing increases recall, with a caveat

The cognitive interview, developed by Fisher and Geiselman for witness interviewing, uses techniques such as reinstating context, encouraging the interviewee to report everything, and recalling from different perspectives or orders. Memon, Meissner and Fraser’s meta-analysis, “The Cognitive Interview: A meta-analytic review and study space analysis of the past 25 years” (Psychology, Public Policy, and Law, 2010), found it elicited substantially more correct details than comparison interviews, with a small increase in incorrect details. More recall comes with some errors, which is a good reason to check every claim afterwards.

Step 1: Prepare before you talk

Read what exists: datasheets, application notes, design documents, support tickets. Write the questions the content’s reader will have. Send the expert a short note on the audience and the purpose so they can bring the right examples.

Step 2: Ask about a specific case

Instead of “how does the calibration work?”, ask “tell me about a time calibration went wrong in the field” or “walk me through the last unit you brought up on the bench.” Specific cases surface details that general questions don’t.

Step 3: Go through the case more than once

First let the expert tell the story. Then build a timeline together. Then go back to each decision point. Each pass adds detail the previous one missed.

Step 4: Probe the cues and the judgments

At each decision point, ask what they noticed, what they were trying to achieve, what else they considered and what they expected to happen. “How did you know?” is often the most productive question in the interview.

Step 5: Ask what a novice would miss

Ask directly: “What would someone new to this get wrong?” and “What do you check that the datasheet doesn’t tell you to?” These questions work against the curse of expertise.

Step 6: Have them show or sketch it

Ask the expert to draw the signal path, point at the board, share their screen or run the measurement. Showing exposes steps that description skips.

Step 7: Record, then send the claims back

Record with permission and transcribe. Turn the transcript into a list of specific claims and numbers, and send that list, not the transcript, back to the expert for correction. Flag anything that came from memory rather than a document or measurement.

Step 8: Interview a second person

Talk to someone closer to the reader, such as an applications or field engineer. The design engineer knows what’s true; the applications engineer knows what confuses people.

Interview questions that work, and ones that don’t

Instead of	Ask	Why
“Tell me about the product.”	“Walk me through the last time you set one up.”	Specific cases surface real detail
“How does it work?”	“Draw the signal path and talk me through it.”	Showing exposes skipped steps
“What are the benefits?”	“What problem did this solve that the old approach didn’t?”	Gets the mechanism, not the slogan
“Is it easy to use?”	“What do new users get wrong?”	Counters the curse of expertise
“What’s the accuracy?”	“Under what conditions, and how was it measured?”	Gets checkable numbers
“Anything else?”	“What would make you change your mind about this design?”	Reveals limits and trade-offs

How to put it to work

If you have an hour

Rewrite your standard interview guide using the right-hand column of the table, and add the “what would a novice miss” question at the end.

If you have a sprint

Run one interview with the case-based, multi-pass structure, turn it into a claims list, and send it back for review. Compare the result with a past interview on a similar topic.

If you want it to last the product’s lifetime

Keep the interview recordings, claims lists and expert corrections with the content, so future updates start from verified material. In our radiology-physics animation work for Sybil Digital Learning, the educators provided the subject expertise and reviewed the explanations, and that review loop is what this step is for.

The question to ask yourself: “Did I get the expert’s judgment, or just their summary?”

Frequently asked questions

How do you interview a subject matter expert?

Prepare by reading existing material, ask about a specific real case, go through it several times at increasing depth, probe the cues and decisions behind each step, have the expert show or sketch, and send the resulting claims back for review.

What questions should I ask a technical expert?

Questions about specific situations (“walk me through the last time…”), about cues (“how did you know?”), about novices (“what do new users get wrong?”) and about limits (“what would make you change your mind?”).

Why do experts leave out important details?

Experts often can’t fully report their own reasoning and tend to underestimate what novices don’t know. Steps that feel obvious to them are skipped without them noticing.

Should I send the transcript to the expert for review?

Send a list of specific claims and numbers instead. It’s faster to review and makes errors easier to spot than a long transcript.

How long should an expert interview be?

Long enough to go through one or two cases in depth, often under an hour, with a short follow-up for questions that come up while writing.

Related reading: from expert interview to technical explainer, writing technical articles engineers trust and briefing a content partner for an embedded launch.

If you’d like your experts interviewed this way and their knowledge turned into reviewed technical content, that’s what we do at AlphaCrew Studio.

Sources

1. Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84(3), 231–259. https://doi.org/10.1037/0033-295X.84.3.231
2. Hinds, P. J. (1999). The curse of expertise: The effects of expertise and debiasing methods on prediction of novice performance. Journal of Experimental Psychology: Applied, 5(2), 205–221. https://doi.org/10.1037/1076-898X.5.2.205
3. Klein, G. A., Calderwood, R., & MacGregor, D. (1989). Critical decision method for eliciting knowledge. IEEE Transactions on Systems, Man, and Cybernetics, 19(3), 462–472. https://doi.org/10.1109/21.31053
4. Hoffman, R. R., Shadbolt, N. R., Burton, A. M., & Klein, G. (1995). Eliciting knowledge from experts: A methodological analysis. Organizational Behavior and Human Decision Processes, 62(2), 129–158. https://doi.org/10.1006/obhd.1995.1039
5. Memon, A., Meissner, C. A., & Fraser, J. (2010). The Cognitive Interview: A meta-analytic review and study space analysis of the past 25 years. Psychology, Public Policy, and Law, 16(4), 340–372. https://doi.org/10.1037/a0020518