With PyTorch 1.5, the RPC framework can be used to build training applications that make use of distributed architectures if they’re available.

Aaron Saxton from NCSA gave this talk at the Blue Waters Symposium. "Blue Waters currently supports TensorFlow 1.3, PyTorch 0.3.0 and we hope to support CNTK and Horovod in the near future. This ...

April 7, 2022 — The Habana team announced it is collaborating with Grid.ai to make it easier for developers to train on Gaudi processors with PyTorch Lightning without code changes. Grid.ai and ...

Soumith Chintala, PyTorch project lead, seems to share Zaharia's ideas about distributed training being the next big thing in deep learning, as it has been introduced in the latest version of PyTorch.