The Lambda
Deep Learning Blog

Published on February 18, 2020 by Michael Balaban

State-of-the-art (SOTA) deep learning models have massive memory footprints. Many GPUs don't have enough VRAM to train them. In this post, we determine which ...

Published on December 19, 2019 by Chuan Li

by Chuan Li, PhD

Published on October 30, 2019 by Stephen Balaban

The desired end-state of this tutorial, a running subnet manager on your switch. This tutorial will walk you through the steps required to set up a Mellanox ...

Published on October 1, 2019 by Chuan Li

TensorFlow 2 is now live! This tutorial walks you through the process of building a simple CIFAR-10 image classifier using deep learning. In this tutorial, we ...

Published on August 28, 2019 by Chuan Li

This blog will walk you through the steps of setting up a Horovod + Keras environment for multi-GPU training.

Published on August 12, 2019 by Chuan Li

One of the most asked questions we get at Lambda Labs is, “how do I track resource utilization for deep learning jobs?” Resource utilization tracking can help ...

Published on June 7, 2019 by Chuan Li

Distributed training allows scaling up deep learning task so bigger models can be learned or training can be conducted at a faster pace. In a previous ...

Published on June 6, 2019 by Chuan Li

During training, weights in the neural networks are updated so that the model performs better on the training data. For a while, improvements on the training ...

Published on June 6, 2019 by Chuan Li

This tutorial combines two items from previous tutorials: saving models and callbacks. Checkpoints are saved model states that occur during training. With ...

Published on June 5, 2019 by Chuan Li

This tutorial shows you how to perform transfer learning using TensorFlow 2.0. We will cover:

Published on May 31, 2019 by Stephen Balaban

This presentation is a high-level overview of the different types of training regimes that you'll encounter as you move from single GPU to multi GPU to multi ...

Published on March 12, 2019 by Michael Balaban

In this post, Lambda Labs benchmarks the Titan V's Deep Learning / Machine Learning performance and compares it to other commonly used GPUs. We use the Titan V ...

Published on March 4, 2019 by Stephen Balaban

by Chuan Li, PhD

Published on February 17, 2019 by Stephen Balaban

CPU, GPU, and I/O utilization monitoring using tmux, htop, iotop, and nvidia-smi. This stress test is running on a Lambda GPU Cloud 4x GPU instance. Often ...

Published on February 16, 2019 by Stephen Balaban

Update June 5th 2020: OpenAI has announced a successor to GPT-2 in a newly published paper. Checkout our GPT-3 model overview.