Your machine learning team is building and planning to operationalize a machine learning model that uses deep learning to recognize and classify images of potential security threats to important government officials using stills taken from live security video. However, when your team runs their mini-batch training of the neural network, the training accuracy oscillates over your training epochs.
What is the most probable cause of your training accuracy problem?

Answer options:

A.The validation error has stopped decreasing.
B.The mini-batch size is too small.
C.The mini-batch size is too large.
D.The learning rate is very high.

Answer: D
Option A is incorrect. In deep learning training, choosing the correct number of epochs is important. The validation error is used to determine how many epochs to run through. When the learning rate stops decreasing, you should stop running training epochs. The point when your validation error stops decreasing has no correlation to the oscillation of the accuracy of your training epochs.
Option B is incorrect. A small mini-batch is used to prevent your training process from stopping at local minima. Having a small mini-batch size won’t cause oscillation in your training epoch accuracy.
Option C is incorrect. A large mini-batch size is used to allow for highly computational demanding matrix multiplication in your training calculations. Having a large mini-batch size won’t cause oscillation in your training epoch accuracy.
Option D is CORRECT. A very high learning rate tends to cause oscillation in your training accuracy. A high learning rate causes your weight updates to be too large, and you will overestimate your goal and oscillate around the true goal.
Reference:
Please see the Amazon SageMaker developer guide titled DeepAR Forecasting Algorithm.
Please refer to the Machine Learning Mastery article titled How to Configure the Learning Rate When Training Deep Learning Neural Networks.
Please review the article titled Hyperparameters in Machine /Deep Learning.
Please refer to the Towards Data Science article titled Hyper-parameter Tuning Techniques in Deep Learning.