PyTorch tensor objects for storing and updating model parameters
We covered tensor objects in Chapter 12, Parallelizing Neural Network Training with PyTorch. In PyTorch, a special tensor object for which gradients need to be computed allows us to store and update the parameters of our models during training. Such a tensor can be created by just assigning requires_grad to True on user-specified initial values. Note that as of now (mid-2021), only tensors of floating point and complex dtype can require gradients. In the following code, we will generate tensor objects of type float32:
>>> a = torch.tensor(3.14, requires_grad=True)
>>> print(a)
tensor(3.1400, requires_grad=True)
>>> b = torch.tensor([1.0, 2.0, 3.0], requires_grad=True)
>>> print(b)
tensor([1., 2., 3.], requires_grad=True)
Notice that requires_grad is set to False by default. This value can be efficiently set to True by running requires_grad_().
method_() is an in...
