A Mathematical Framework for Transformer Circuits

Source: Anthropic / Transformer Circuits Date: 2021-01-01

Summary

This essay reframes transformers as collections of small computational motifs that read from and write to a shared residual stream. It is still the clearest public articulation of the residual stream as the model’s central communication bus.

Why it matters here

For NNPL, this is the conceptual bridge from “token model” to “editable workspace.” If the residual stream really is the shared substrate through which heads and MLPs communicate, it becomes plausible to attach read/write interfaces there.