Multi-GPU enabled BERT using Horovod
BERT is Google's pre-training language representations which obtained the state-of-the-art results on a wide range of Natural Language Processing tasks. ...
Featured
Introducing NVIDIA SHARP on Lambda 1CC: Next-Gen Performance for Distributed AI Workloads
Lambda’s 1-Click Clusters(1CC) provide AI teams with streamlined access to scalable, multi-node GPU clusters, cutting through the complexity of distributed infrastructure. Now, we're pushing the envelope further by integrating NVIDIA's Scalable Hierarchical Aggregation and Reduction Protocol (SHARP) into our multi-tenant 1CC environments. This technology reduces communication latency and improves bandwidth efficiency, directly accelerating training speed of distributed AI workloads.
Recent
On-prem GPU Training Infrastructure for Deep Learning - Slides
These slides are from my talk at Rework Deep Learning Summit 2019.
Reproduce Fast.ai/DIUx imagenet18 with a Titan RTX server
Last, tyear, Fast.ai won the first ImageNet training cost challenge as part of the DAWN benchmark. Their customized ResNet50 takes 3.27 hours to reach 93% ...
To top