The perceptron is a type of simple neural network that can be used for classification through supervised learning. The network is trained with examples, and then the weights are adjusted based on the actual output vs. the expected result.
Frank Rosenblatt invented the first perceptron, simulating it on an IBM® 704 computer before developing it as custom hardware using 400 photocells for vision applications.
The weights were implemented as potentiometers (variable resistors) that associated motors could modify as part of the learning process, and the photocells were randomly connected to neurons.
A simple perceptron with two inputs (and related weights) and a bias weight is shown in the graphic below.
The perceptron works by adding up the products of the inputs and their weights, then applying the result via an activation function. The activating function, in this case, is a step function, which states that if the output is greater than 1, the result is 1. (otherwise, the outcome is 0).
As seen in the following image, the simple perceptron can solve linearly separable problems.
In this picture, a line divides the two classes (the result of a logical OR operation), which can be shown as a straight line (or decision boundary).
The weights for the inputs and the bias determine the decision boundary.
The OR and AND issues are linearly separable; however, the XOR problem isn’t (given 1 XOR 1 is 0 and not detachable).
Let’s look at how you “teach” the perceptron through supervised training now that you have a better understanding of the challenges perceptrons can tackle.