When would you use GD over SDG, and vice-versa?

Data Science, AI and ML
, ,
#1

Use of GD over SDG and vice versa

#2

GD theoretically minimizes the error function better than SGD. However, SGD converges much faster once the dataset becomes large.

That means GD is preferable for small datasets while SGD is preferable for larger ones.

In practice, however, SGD is used for most applications because it minimizes the error function well enough while being much faster and more memory efficient for large datasets.

Data Preprocessing