Setting up Horovod + Keras for Multi-GPU training
This blog will walk you through the steps of setting up a Horovod + Keras environment for multi-GPU training.
Featured
Introducing NVIDIA SHARP on Lambda 1CC: Next-Gen Performance for Distributed AI Workloads
Lambda’s 1-Click Clusters(1CC) provide AI teams with streamlined access to scalable, multi-node GPU clusters, cutting through the complexity of distributed infrastructure. Now, we're pushing the envelope further by integrating NVIDIA's Scalable Hierarchical Aggregation and Reduction Protocol (SHARP) into our multi-tenant 1CC environments. This technology reduces communication latency and improves bandwidth efficiency, directly accelerating training speed of distributed AI workloads.
Recent
Tracking system resource (GPU, CPU, etc.) utilization during training with the Weights & Biases Dashboard
One of the most asked questions we get at Lambda Labs is, “how do I track resource utilization for deep learning jobs?” Resource utilization tracking can help ...
TensorFlow 2.0 Tutorial 05: Distributed Training across Multiple Nodes
Distributed training allows scaling up deep learning task so bigger models can be learned or training can be conducted at a faster pace. In a previous ...
To top