000263719 001__ 263719
000263719 005__ 20190619220202.0
000263719 037__ $$aCONF
000263719 245__ $$aSelf-Binarizing Networks
000263719 260__ $$c2019-02-05$$barXiv
000263719 269__ $$a2019-02-05
000263719 300__ $$a11
000263719 336__ $$aConference Papers
000263719 520__ $$aWe present a method to train self-binarizing neural networks, that is, networks that evolve their weights and activations during training to become binary. To obtain similar binary networks, existing methods rely on the sign activation function. This function, however, has no gradients for non-zero values, which makes standard backpropagation impossible. To circumvent the difficulty of training a network relying on the sign activation function, these methods alternate between floating-point and binary representations of the network during training, which is sub-optimal and inefficient. We approach the binarization task by training on a unique representation involving a smooth activation function, which is iteratively sharpened during training until it becomes a binary representation equivalent to the sign activation function. Additionally, we introduce a new technique to perform binary batch normalization that simplifies the conventional batch normalization by transforming it into a simple comparison operation. This is unlike existing methods, which are forced to the retain the conventional floating-point-based batch normalization. Our binary networks, apart from displaying advantages of lower memory and computation as compared to conventional floating-point and binary networks, also show higher classification accuracy than existing state-of-the-art methods on multiple benchmark datasets.
000263719 6531_ $$aNeural networks, Binarization, Optimization
000263719 700__ $$g254809$$aLahoud, Fayez$$0254717
000263719 700__ $$g172126$$aAchanta, Radhakrishna$$0242495
000263719 700__ $$aMarquez Neila, Pablo$$0248120$$g243370
000263719 700__ $$0241946$$aSüsstrunk, Sabine$$g125681
000263719 8560_ $$ffayez.lahoud@epfl.ch
000263719 8564_ $$uhttps://infoscience.epfl.ch/record/263719/files/TanhNet.pdf$$s334954
000263719 909C0 $$xU10429$$pIVRL$$msabine.susstrunk@epfl.ch$$zGrolimund, Raphael$$0252320
000263719 909CO $$qGLOBAL_SET$$pconf$$pIC$$ooai:infoscience.epfl.ch:263719
000263719 960__ $$afayez.lahoud@epfl.ch
000263719 961__ $$afantin.reichler@epfl.ch
000263719 973__ $$aEPFL
000263719 980__ $$aCONF
000263719 981__ $$aoverwrite