Posit Arithmetic Units for Deep Neural Networks

Abstract

Posit™ arithmetic is a recent alternative format to the IEEE 754 standard for floating-point numbers that claims to provide compelling advantages over floats, including higher accuracy, larger dynamic range or bitwise compatibility across systems. In particular, this format is a suitable candidate to replace floating-point numbers in Deep Neural Networks (DNNs), an area of growing interest with a large computational cost. This work presents parameterized designs for multiple posit functional units, including addition, multiplication and multiply-accumulate operation, and integrate them as templates of the FloPoCo framework. Synthesis results show that the proposed arithmetic units significantly reduce the hardware requirements when compared with previous implementations. Finally, this work proposes the use of posit arithmetic for performing both DNN inference and training. Experiments on different datasets, including CIFAR-10, reveal that 16-bit posits can safely replace 32-bit floats for training, and that low-precision 8-bit posits can be used for DNN inference with negligible accuracy drop.