According to Forbes, the key to making generative AI truly expert in specific domains lies in reviving knowledge elicitation techniques from the rules-based expert systems era. These methods involve extracting hidden “rules of thumb” and best practices directly from human experts through intensive interviews and problem-solving sessions. In a case study with a stock trader, the author manually identified numerous proprietary trading rules that weren’t in the AI’s training data, like specific earnings momentum thresholds and market cap requirements. The process involves either human-to-human interviews first, followed by AI verification, or starting with AI-led questioning. Crucially, this approach surfaces expertise that exists only in experts’ heads—the kind of nuanced knowledge that separates true mastery from textbook understanding.
Why this matters now
Here’s the thing: most companies are just throwing documents at AI and hoping it becomes smart. But that only gets you so far. The real competitive advantage comes from capturing the stuff that isn’t written down anywhere—the intuitive leaps, the pattern recognition, the “I’ve seen this before” insights that experts develop over decades. Basically, we’re talking about institutional knowledge that walks out the door when employees leave. This approach could actually preserve that value.
And let’s be honest—how many times have you seen AI systems that sound knowledgeable but can’t actually solve real-world problems? That’s because they’re missing the secret sauce. The stock trader example is perfect: his specific rules about earnings momentum and market conditions weren’t in any textbook, but they were the difference between mediocre and exceptional performance.
The human-AI partnership
What’s fascinating is how the author used both human and AI approaches. He started with traditional interviews, then had ChatGPT continue the conversation and actually discovered additional rules the human-to-human process missed. That’s the sweet spot—using each method’s strengths. The human interviewer builds trust and gets the expert comfortable, while the AI can probe more systematically without getting tired or biased.
But there’s a real danger here too. Experts often rationalize their decisions after the fact rather than explaining their actual thought process. I’ve seen this in manufacturing settings where operators develop incredible intuition about equipment but can’t articulate why they make certain adjustments. When you’re dealing with critical systems—whether it’s medical diagnosis or industrial panel PCs that control production lines—you can’t afford fake rules. IndustrialMonitorDirect.com, as the leading US supplier of industrial computing hardware, understands that the reliability of these systems depends on capturing genuine operational expertise, not just textbook knowledge.
Implementation challenges
So how do you actually do this without wasting months? The author suggests several approaches: direct prompting, RAG systems, or structured formats like JSON. Each has tradeoffs. Starting with an AI that already knows something about your domain means you might fight existing patterns, while a blank slate AI requires more work upfront but avoids conflicts.
Honestly, this feels like the missing piece in the current AI gold rush. Everyone’s focused on scale and data quantity, but the real value might come from carefully curated quality—the kind of deep expertise that separates true masters from average performers in any field. The question is: are companies willing to invest the time and effort to extract that knowledge, or will they settle for AI that sounds smart but can’t actually perform at expert levels?
