Discussion Forum | Board Infinity

When would you use GD over SDG, and vice-versa?

Use of GD over SDG and vice versa

GD theoretically minimizes the error function better than SGD. However, SGD converges much faster once the dataset becomes large.

That means GD is preferable for small datasets while SGD is preferable for larger ones.

In practice, however, SGD is used for most applications because it minimizes the error function well enough while being much faster and more memory efficient for large datasets.

Data Preprocessing