Stochastic Gradient Descent (SDA)
In terms of artificial neural networks, an epoch refers to one cycle through the full training dataset. Usually, training a neural network takes more than a few epochs. In other words, if we feed a neural network the training data for more than one epoch in different patterns, we hope for a better generalization when given a new “unseen” input (test data). An epoch is often mixed up with an iteration. Iterations is the number of batches or steps through partitioned packets of the training data, needed to complete one epoch. Heuristically, one motivation is that (especially for large but finite training sets) it gives the network a chance to see the previous data to readjust the model parameters so that the model is not biased towards the last few data points during training.
One Epoch is when an ENTIRE dataset is passed forward and backward through the neural network only ONCE.
Since one epoch is too big to feed to the computer at once we divide it in several smaller batches.
Ref: https://towardsdatascience.com/epoch-vs-iterations-vs-batch-size-4dfb9c7ce9c9
Ref: https://deepai.org/machine-learning-glossary-and-terms/epoch
Epoch is nothing but a hyperparameter that , in simple terms, iteration constituting one forward pass and one backward pass (preferably in neural network)
Since Epoch cannot be applied to a very large dataset, it is divided to form a batches . Iteration is the number of batches needed to complete one epoch.
EXAMPLE: If we have 5000 samples in a dataset ,if we need to find epoch, let’s suppose there are 500 batches (assume), then there would be 10 iterations to complete 1 epoch.