Know Thyself: Evaluating Self-prediction Capabilities in LM Agents

As LM agents are increasingly deployed in long-horizon, high-stakes settings, their ability to predict their own behavior becomes safety-relevant: accurate self-prediction could power introspection-based early-alert systems, yet the same capability may also be a prerequisite for sophisticated misaligned behavior. We introduce a plug-and-play pipeline for measuring and tracking this dual-use capability in agents across three stages: benign self-prediction on agentic software-engineering tasks (SWE-bench Verified), automated auditing built on the Petri framework, and an adversarial AI R&D setting that probes for scheming in conjunction with self-prediction. Our preliminary results show that current models are poor self-predictors. On SWE-bench, forecasters are systematically miscalibrated, and no ablation overcomes the deficit. In agentic audits we find a genuine but narrow self-knowledge signal, while in the adversarial setting models chronically over-predict scheming and, under a safety-audit framing, selectively deny their own. Miscalibration is the dominant failure mode throughout. More robust claims will require larger model populations and richer self-prediction tasks, which our tooling is designed to enable.