Out of Context Obfuscation: What Facts Matter for Probe Evasion

We ask whether language models can evade white-box monitors solely by leveraging factual knowledge of how those monitors work and where they fail.  To measure this, we start with a malign initialization, where the LM is trained to hide adverse drug findings in a clinical reporting task.  We then teach the model various facts about a fictitious white-box monitoring system within this environment, and evaluate whether teaching these facts results in a drop in a lie-detection probe's recall.  Across Qwen-3-14B, -32B, and Llama-3.3-70B-Instruct, we find that language models successfully evade probes only when they are taught a concrete evasion strategy and given a strong incentive to deploy it — removing either collapses the effect.  Because this type of evasion requires no optimization and only knowledge of how the probe fails, a potentially scheming model that already has the propensity to do so could plausibly acquire the same capability through pretraining alone, supporting calls to filter such content from pretraining corpora. Our code and datasets can be viewed here: https://anonymous.4open.science/r/out-of-context-probe-obfuscation-62F7/