Optimizing Transformer Architectures for Ultra-Low Latency Inference in Mobile Devices

Abstract

Deep neural networks deployed on resource-constrained edge devices require compact architectures. In this paper, we propose a lightweight transformer encoder with quantized self-attention mechanisms that reduces memory bandwidth overhead by 42% while retaining 98.5% top-1 accuracy on standard benchmarking datasets.