The cost of local Transformer learning.
What happens when a Transformer learns in separate stages? Two CPU studies examine the speed, memory, and learning quality of local training and predictive coding.
Learning has a cost.
Two separate CPU studies
Where should the learning signal go?
Local stages can train in parallel. Recovering information from later stages takes additional work.
24-layer Transformer · Prespecified server study
Flow shown schematically. RGC uses 29 stage-worker threads versus 24 for backpropagation. The selected broadcast method failed the held-out 1% quality criterion. Source: the submitted Machine Learning manuscript.
What the study shows.
Parallelism has a memory cost.
In the 24-layer server study, asynchronous RGC reaches 1.382× the throughput of backpropagation while process-tree memory rises from 1.94 to 4.31 GiB. It uses 29 worker threads versus 24.
Recovering credit takes computation.
In a separate 12-layer study, three predictive-coding inference rounds yield a +0.142% mean paired validation-loss gap at 2.78× the serial training time of backpropagation.
Quality remains the deciding test.
The selected broadcast method does not pass the held-out 1% non-inferiority criterion. The predictive-coding experiment is exploratory and does not revise that result.
Scope & details.
Local Credit Assignment for CPU Transformers: Readout Consensus and the Cost of Predictive Coding
DOI 10.21203/rs.3.rs-11059245/v1CC BY 4.0
The two studies use different models, hardware, and evaluation settings. Their ratios have separate backpropagation baselines and should not be pooled. These experiments do not establish a general replacement for backpropagation.
Read the abstract
Local objectives expose parallelism across Transformer depth but remove downstream credit. We study this trade-off in two complementary experiments on central processing units. A prospectively specified campaign on 24-layer byte-level Transformers evaluates readout-gradient consensus (RGC), which coordinates shared-readout parameter gradients without restoring cross-stage activation gradients. Asynchronous RGC reaches 1.382 times backpropagation throughput, while process-tree memory rises from 1.94 to 4.31 GiB and active worker threads increase from 24 to 29. The selected broadcast method fails its held-out 1% non-inferiority criterion; TinyStories transfer and the primary readout-equivalence criterion also fail. A separate, exploratory 27-run study applies stage-boundary augmented Lagrangian predictive coding (PC-ALM) to a smaller 12-layer Transformer. Three inference rounds produce a mean paired validation-loss gap of +0.142% versus backpropagation at 2.78 times its serial training time. Initialising multipliers from local readout gradients degrades learning; penalty-only predictive coding retains much of the quality benefit. The evidence identifies two distinct costs: quality lost by truncating credit and computation spent recovering it. It establishes neither quality-preserving CPU acceleration nor a general replacement for backpropagation.
Citation
@unpublished{lex2026training,
title={Local Credit Assignment for CPU Transformers: Readout Consensus and the Cost of Predictive Coding},
author={Lex, Vikram},
year={2026},
doi={10.21203/rs.3.rs-11059245/v1},
url={https://doi.org/10.21203/rs.3.rs-11059245/v1},
note={Manuscript under review at Machine Learning}
}