Essays about: "Ring all-reduce."

Found 2 essays containing the words Ring all-reduce..

1. 1. Distributed Robust Learning

   University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

   Author : Akhil Yerrapragada; [2021]
   Keywords : Byzantine resilient decentralized training; Gradient aggregation rules; α; f Byzantine resilience; Fault tolerance; Ring all-reduce.; Byzantinsk motståndskraftig decentraliserad träning; Gradientaggregeringsregler; α; f Byzantinsk motståndskraft; Feltolerans; Ring allreducera.;

   Abstract : Accuracy obtained when training deep learning models with large amounts of data is high, however, training a model with such huge amounts of data on a single node is not feasible due to various reasons. For example, it might not be possible to fit the entire data set in the memory of a single node, training times can significantly increase since the dataset is huge. READ MORE

3. 2. Analysis and Comparison of Distributed Training Techniques for Deep Neural Networks in a Dynamic Environment

   University essay from KTH/Skolan för elektroteknik och datavetenskap (EECS)

   Author : Ermias Gebremeskel; [2018]
   Keywords : deep learning; large scale distributed deep learning; data parallelism;

   Abstract : Deep learning models' prediction accuracy tends to improve with the size of the model. The implications being that the amount of computational power needed to train models is continuously increasing. Distributed deep learning training tries to address this issue by spreading the computational load onto several devices. READ MORE