
Rethinking the Tradeoff Between Temporal Encoding and Nonlinear Computation in Spiking Language Models
Spora replaces conventional floating-point activations and nonlinear attention in spiking language models with binary temporal weight encodings (UBS/BBS) that support accumulation-and-shift dot products and integer-exponent attention mappings. The cost is a full custom training pipeline with spike encoding, thresholds, residual decay, and fixed-point arithmetic, plus added time-step computation, with quality still below mainstream quantized Transformers at comparable scale. The approach can fail in practice because its efficiency claims depend on neuromorphic or fixed-point event-driven hardware that a standard 24 GB GPU does not exploit, tooling is immature, and validation is limited to GLUE-scale encoder tasks rather than production generative workloads.







































