Layers > Dense

What is it

A neural network is made out of layers. The dense layer is also called the fully connected layer or just known as a 'normal' layer. It is composed of weights connecting each input with every output, and then it adds the biases to each output.

The Dense layer is complementary with the Activation layer as the dense layer is the part that will train and improve and the activation layer produces non-linearity or said in another way: it allows the network to learn many more things.

How to create a Dense layer

To create a layer we first need a neural network to create the layer in, as seen in the XOR example.

We can then call the add_layer method on the neural network and create the layer that we want as the parameter for the function. In our case we are going to create a Dense layer.

The first two arguments are for the number of inputs and the number of outputs of the layer, because as said earlier each input is connected with every output and there is a bias for each output. The third ad fourth arguments are the range for initialisation of weights and biases respectively. In this example for weights, they can be initialized between -0.7 to 0.7.

model.add_layer(nn.Dense.new(2, 3, 0.7, 0.65))