Hi,

Today you’re going to learn how to train your first neural network using the PyTorch library:

Image

The big picture: We all need to start somewhere — welcome to your intro to PyTorch. This tutorial will teach you the fundamentals of training a neural network with PyTorch, including:

  1. How to define a basic neural network architecture with PyTorch
  2. How to define your loss function and optimizer
  3. How to properly zero your gradient, perform backpropagation, and update your model parameters â€” most deep learning practitioners new to PyTorch make a mistake in this step 

How it works: If you’re already familiar with Keras/TensorFlow, then when going through today’s tutorial, you may be surprised how much PyTorch code is required to train a simple neural network, predominantly in the training loop itself.

While Keras/TensorFlow encapsulates the training procedure into a single â€œmodel.fit” call, PyTorch allows you full control and customizability over the training procedure — the PyTorch developers designed the library in this way.

Advanced TensorFlow users familiar with â€œGradientTape” know that it takes significantly more code to train a network than simply calling â€œmodel.fit”, but the additional control you have is sometimes worth it, especially if you are doing state-of-the-art research.

Thus, you think of PyTorch as being in â€œGradientTape” mode by default â€” the training loop must be implemented by hand which will require additional code.

My thoughts: Yes, coding neural networks and training procedures with PyTorch does take more code, and oftentimes more effort than the higher-level Keras API; however, Keras doesn’t give you control over what happens inside â€œmodel.fit” â€” PyTorch does.

Yes, but: All that flexibility comes at a cost though. I haven’t met a single deep learning practitioner who hasn’t at least once screwed up:

  1. Zeroing their gradients with â€œopt.zero_grad()”
  2. Performing backpropagation with â€œloss.backward()”
  3. Updating their model weights with â€œopt.step()”

Don’t know what I’m talking about?

Well, to quote master Yoda…

Image

Stay smart: Take your time learning the basics with PyTorch. While the library is incredibly customizable, it tends to have a higher initial learning curve than Keras/TensorFlow.

This series of tutorials on PyTorch is meant to ramp you up as quickly as possible and get you to the point where you see neural networks like Neo does The Matrix:

Image

Click here to learn how to train your first neural network with PyTorch.


Adrian Rosebrock
Chief PyImageSearcher

P.S. If you're interested in learning how to successfully apply deep learning to your own projects, I would recommend reading my book, Deep Learning for Computer Vision with Python.

I crafted my book so that it perfectly balances theory with implementation, ensuring you properly master:
 
  • Deep learning fundamentals and theory without unnecessary mathematical fluff. I present the basic equations and back them up with code walkthroughs that you can implement and easily understand. You don’t need a degree in advanced mathematics to understand this book
  • How to implement your own custom neural network architectures. Not only will you learn how to implement state-of-the-art architectures, including ResNet, SqueezeNet, etc., but you’ll also learn how to create your own custom CNNs
  • How to train CNNs on your own datasets. Most deep learning tutorials don’t teach you how to work with your own custom datasets. Mine do. You’ll be training CNNs on your own datasets in no time
  • Object detection (Faster R-CNNs, Single Shot Detectors, and RetinaNet) and instance segmentation (Mask R-CNN). Use these chapters to create your own custom object detectors and segmentation networks
You’ll also find answers and proven code recipes to:
  • Create and prepare your own custom image datasets for image classification, object detection, and segmentation
  • Hands-on tutorials (with lots of code) that not only show you the algorithms behind deep learning for computer vision but their implementations as well
  • Put my tips, suggestions, and best practices into action, ensuring you maximize the accuracy of your models
If you’re interested in learning more about my deep learning book, I’d be happy to send you a free PDF containing the Table of Contents and a few sample chapters:

Click here to download your table of contents and sample chapters PDF

After clicking the link above, the PDF will land in your inbox in a few short minutes.