Training > Backpropagation

What is it

Backpropagation is a commonly used training algorithm for neural networks.

Parameters

We need to specify different parameters to be able to make the neural network train successfully.

The learning_rate is a factor that controls the speed of the adjustements of the weights and biases. If it is high the training can be faster but it can overshoot too. If it is low the learning will take more time but can be more precise and get closer to the solution more safely.

The momentum is a factor that controls how much of the precedent adjustement is applied with the new adjustement to the weights and biases. It allows the network to converge a lot faster in a lot of cases as the modification will get more and more momentum towards the solution and less towards noise from the data.

The batch_size is how many items of a dataset will be used to create each batch. A batch is a collection of items that the neural network will go through and then apply the changes calculated during these items's backpropagation. And then it will go through the next batch and repeat.

The nb_epochs is how many epochs the learning will last. An epoch is finished when the neural network has seen all the dataset one time.

classifier tells the training algorithm if it needs to keep track of the accuracy of the classification and other convenient things for classifiers.

The neural newtork will print it's performance every print_batch_interval batchs on every print_interval epochs.

The cost_function is the cost function that you choose for this training.

training and test are the dataset that are used respectively for the training and the tests.

test_params allows finer control over frequency of the tests and what is displayed during the tests.

pub struct MinibatchesBackpropTrainingParams { learning_rate f64 momentum f64 batch_size int = 1 nb_epochs int classifier bool print_interval int print_batch_interval int cost_function CostFunctions training Dataset test Dataset test_params TestParams }