Understanding Gradient from absolute scratch using PyTorch

Gradient is most common used term in Machine Learning and it is quite confusing for beginners. In this article I will try to explain what is gradient using PyTorch from absolute scratch.

Lets take very simple example of equation of a line.

y = x

This is equation of a line which passes through origin(where x=0 and y =0)

Now lets calculate rate of change of y with respect to change in x. This is also called slope or gradient. Lets use simple mathematics for derivation.

y = x

dy / dx = d / dx (x)

dy / dx = 1

From above very simple derivation we can see rate of change of y with respect of change in x is 1. We can say gradient is 1 here. Now we will try to calculate same using PyTorch

import torch

x  = torch.randn(1, requires_grad=True)
print(x)
y = x
print(y)
y.backward()
print(x.grad)
Output:
tensor([-0.4385], requires_grad=True)
tensor([-0.4385], requires_grad=True)
tensor([1.])

From above piece of code we can see gradient of x is 1 which is same what we calculated above with mathematics

You can try to calculate gradient for equation y = x*x using mathematics and then using PyTorch and match output of both approach as exercise.

Hope this article improved your understanding of gradient in machine learning.