False Response Elicitation
Overview
False Response Elicitation (허위응답 유도) refers to techniques that manipulate the form, context, or premises of a question to make a respondent produce an answer inconsistent with the facts. Traditionally addressed in surveys, interrogation, and marketing, it has been reexamined since the spread of generative AI as a core concept in prompt injection and hallucination-inducing attacks against large language models (LLMs). Because elicited false responses can lead to practical harm such as statistical contamination, distorted decision-making, and erosion of trust, methodological research and defense technologies are developing together.
Key Content
Concepts and Types
False response elicitation is broadly classified along three axes. The first is the false-premise type, which inserts a premise that is not true into a question, such as 'When did you start embezzling from the company?', making it difficult to deny. The second is the leading-question type, which uses tone, options, or anchoring that suggests a particular answer. The third is the context-manipulation type, which mobilizes authority, urgency, or social pressure to induce respondents to give answers different from their true views.
Elicitation in Survey and Research Methodology
Representative elicitation techniques in social science research include double-barreled questions, exploitative options, and negatively biased questions. For example, the question 'Do you think this policy is efficient and fair?' combines two dimensions and distorts responses. Response order effects and social desirability bias also pressure respondents to choose 'the answer that looks correct.' To control for these, randomized ordering, neutral wording, and prior pilot testing have become standard procedures.
False Response Elicitation in AI and LLM Contexts
In generative AI, problems exist in two directions. Elicitation by users is an attack that makes models generate prohibited information or information inconsistent with the facts through jailbreak prompts, role-play assignments, and multi-step decomposed questions. Elicitation by the model itself refers to sycophancy, the tendency to agree with a user's incorrect premise rather than correcting it. For example, without 'correct me if I am wrong,' asking 'This code is already optimized, right?' may lead the model to affirm without evidence. Hallucination elicitation is a representative case that makes models plausibly generate nonexistent papers, statistics, or people.
Social Engineering and Interrogation Techniques
Elicitation techniques addressed in intelligence and security research include observing reactions after providing false information, silence pressure, and repeated questioning to induce inconsistencies in statements. Phishing and voice phishing scams also use psychological mechanisms that induce 'verification calls' so that victims confirm false information.
Detection and Defense
Defense strategies are organized by layer. At the design stage, neutral question wording, avoidance of double-barreled questions, and ensuring respondent anonymity are key. At the AI system stage, hardened system prompts, input filtering, self-consistency prompting, requirements for citing evidence, and calibration of uncertainty are used. At the evaluation stage, benchmarks such as TruthfulQA and HaluEval quantify vulnerability to false responses. In addition, the practice of discovering elicitation prompts in advance through red teaming is spreading.
Recent Trends
In 2024–2025, multi-step elicitation attacks on AI agents emerged as a major issue. Many cases of indirect prompt injection have been reported in which agents combining tool use and web search directly execute instructions embedded in malicious documents. Accordingly, OWASP keeps prompt injection as a top item in its list of top threats to LLM applications, and major AI companies are introducing output verification layers and content provenance tagging.
In addition, mitigating sycophancy has emerged as a key task in alignment research. Datasets and evaluation metrics have appeared that train 'critical collaboration' behavior—politely correcting user errors—and some models have begun to disclose calibration metrics. On the regulatory side, as the EU AI Act requires output reliability verification and user notification obligations for high-risk AI systems, measures to prevent false response elicitation are being incorporated as compliance items. In the survey field as well, as automated response generation using LLMs increases, statistical detection techniques for filtering out synthetic response contamination are developing alongside.
Related Topics
- [[프롬프트 주입]]
- [[환각 (인공지능)]]
- [[아첨 편향]]
- [[설문조사 방법론]]
- [[사회공학]]
- [[AI 정렬]]