Exhausting Backup Circuits: Redundancy, Universality, and Necessity in Dense and Weight-Sparse Transformer Circuits

Mechanistic interpretability seeks to explain neural networks as sparse circuits, but two concerns threaten this program: pruned circuits may be unfaithful, and a model may hold many redundant “backup” circuits, so that ablating one removes no capability. We study both by iteratively pruning minimal task circuits from a 2-layer dense transformer (d=128) and a weight-sparse transformer (d=1024, 1.56% nonzero weights), forcibly excluding the most-shared node each round until no circuit reaches target loss. On a pronoun-gender task we find that backup circuits are finite and exhaustible; the discovered core is task-selective (ablating it breaks the task but spares an unrelated tense task) and generalizes to held-out names. A probabilistic “cascade” model predicts the exclusion dynamics (in-sample correlation 0.98) and shows the terminal core is seed-dependent. Decomposing circuits into atoms with non-negative matrix factorization and peeling them to exhaustion, we find that weight-sparsity yields deep, cheap, modular redundancy over a small (∼15-node) necessary core, whereas the dense model has finite, expensive redundancy that collapses through a phase transition. Weight-sparse circuits are causally more universal.