Response-Aware Defense Against Hidden Malicious Intent in Multi-Turn Dialogue
Xinjie Shen
Mentored by Yiyou Sun, Dawn Song
Working report from the SPAR program. May not reflect the authors' current views.
Abstract
Hidden malicious intent in multi-turn dialogue poses a growing threat to deployed large language models (LLMs). Rather than exposing a harmful objective in a single prompt, increasingly capable attackers can distribute their intent across multiple benign-looking turns. In this work, we address this challenge by detecting the earliest turn at which delivering the candidate response would make the accumulated interaction sufficient to enable harmful action. This objective requires precise turn-level intervention that identifies the harm-enabling ``closure point'' while avoiding premature refusal of benign exploratory conversations. To support training and evaluation, we construct the Multi-Turn Intent Dataset (MTID), which contains branching attack rollouts, matched benign hard negatives, and annotations of the earliest harm-enabling turns. We demonstrate that MTID enables the development of a turn-level monitor, TurnGate, which substantially outperforms existing baselines in harmful-intent detection while maintaining low over-refusal rates.