What structures underpin neural networks in AI?

Neural networks in AI are underpinned by nodes or neurons, weights, biases, and activation functions.

Neural networks, also known as artificial neural networks (ANNs), are a key component of artificial intelligence (AI). They are designed to mimic the human brain's ability to learn and make decisions. The fundamental structures that underpin these networks are nodes or neurons, weights, biases, and activation functions.

Nodes or neurons are the basic units of a neural network. They are inspired by the neurons in the human brain and are where the processing of inputs takes place. Each node receives input from some other nodes, or from an external source and computes an output. Each input has an associated weight (w), which is assigned on the basis of its relative importance to other inputs. The node applies a function (usually non-linear) to the weighted sum of its inputs and produces an output.

Weights are a crucial part of neural networks. They are used to adjust the influence of inputs on a neuron's output. During the training process, these weights are optimised to minimise the difference between the actual and predicted output. The weights are adjusted based on the error at the output and the input to the neuron. This process is known as backpropagation.

Biases, like weights, are also adjustable parameters in a neural network. They are an extra input to neurons and it is always 1, and has its own connection weight. This makes sure that even when all the inputs are none (all 0s) there’s still going to be an activation in the neuron.

Activation functions determine the output of a neural network. These functions are attached to each neuron in the network, and determine whether it should be activated or not, based on whether each neuron’s input is relevant for the model’s prediction. Activation functions also help normalise the output of each neuron to a range between 1 and 0 or between -1 and 1.

In summary, the structures that underpin neural networks in AI are nodes or neurons, weights, biases, and activation functions. These components work together to process inputs and produce outputs, mimicking the way the human brain works.