McCulloch and Pitts developed a computational model for neural networks in the early 1940s, which sparked study into the brain and its application to artificial intelligence.
Later in the decade, Donald Hebb developed Hebbian learning , based on the observation that when two neurons are activated simultaneously, the synapse between them is strengthened.
Frank Rosenblatt invented the perceptron in 1958, a simple neural model that may be used to divide data into two groups. However, this model had a flaw in that it couldn’t categorize an exclusive-OR correctly.
In their book Perceptrons, published in 1969, Marvin Minsky and Seymour Papert exploited this shortcoming to shift the focus of AI back to symbolic approaches. As a result, funding for connectionist research has been declining for a decade.
Paul Werbos developed the back-propagation algorithm in 1975, which successfully trained multilayer perceptrons and introduced a variety of novel multilayer neural network applications. This breakthrough resurrected neural network research and popularised the method for solving real-world challenges.
Neural networks have risen to prominence as a crucial algorithm in machine learning with the introduction of back-propagation.
Large neural networks may now be trained by outsourcing neural network training and execution to clusters of accelerators thanks to the emergence of graphical processing units (GPUs) and distributed computing in recent decades.
Deep learning designs (convolutional neural networks and long short-term memory [LSTM]) emerged, as a result, dramatically expanding the applications and challenges that neural networks can solve.