【ML】Neural network functions

Activation function

1. What is an activation function?

The activation function of a node defines the output of that node given an input or set of inputs.

2. Why do we need an activation function?

It introduces the non-linearities in order to detect non-linear features in the data.

3. Common activation functions

Sigmoid function/curve = logistic function/curve

Loss function

1. What is a loss function?

Measuring the error between the target values t and the output values o: loss(t, o)

2. What is the goal of a loss function?

Find the weights w that minimizes the loss over the training examples

3. Common loss/error function

squared error
log-loss cross-entropy