The work “Parallel and Distributed Training of Deep Neural Networks: A brief overview”, by NEANIAS team, has been published.
The article was published on July 27, 2020, and it is available at IEEE Xplore: 2020 IEEE 24th International Conference on Intelligent Engineering Systems (INES).
- Authors: Attila Farkas [1], Gábor Kertész [1], Róbert Lovas [1].
- Affiliations: [1] Institute for Computer Science and Control (SZTAKI), Hungary.
Abstract
Deep neural networks and deep learning are becoming important and popular techniques in modern services and applications. The training of these networks is computationally intensive, because of the extreme number of trainable parameters and the large amount of training samples. In this brief overview, current solutions aiming to speed up this training process via parallel and distributed computation are introduced. The necessary components and strategies are described from the low-level communication protocols to the high-level frameworks for the distributed deep learning. The current implementations of the deep learning frameworks with distributed computational capabilities are compared and key parameters are identified to help design effective solutions.
Acknowledgments
Ákos Odry, Vladimir Laslo Tadic, Peter Odry, "A Stochastic Logic-Based Fuzzy Logic Controller: First Experimental Results of a Novel Architecture", Access IEEE, vol. 9, pp. 29895-29920, 2021.
Get the whole article at https://eprints.sztaki.hu/9985/.