1 background of artificial neural network
Since ancient times, the mystery about the origin of human intelligence has been attracting the research enthusiasm of countless philosophers and natural scientists. After long-term unremitting efforts, biologists and neuroscientists believe that the intelligent activities of the human brain can not be separated from the material basis of the brain, including its physical structure and various biological, chemical and electrical effects, and thus established the neural network theory and the nervous system structure theory, and the neuron theory is the basis of the nerve conduction theory and the brain function theory. On the basis of these theories, scientists believe that we can study human intellectual activities and cognitive phenomena by imitating the structure and function of human brain nervous system. On the other hand, before19th century, both classical mathematics represented by Euclidean geometry and calculus and classical physics represented by Newtonian mechanics were generally linear sciences. However, the objective world is so complex that nonlinear situations can be seen everywhere, especially in the nervous system of the human brain. Complexity and nonlinearity are linked together, so the study of nonlinear science is also the key to our understanding of complex systems. In order to better understand the objective world, we must study nonlinear science. As a nonlinear network model similar to brain intelligence, artificial neural network came into being. Therefore, the establishment of artificial neural network is not accidental, but the product of the full development of science and technology in the early 20th century.
2. Development of artificial neural network
The research of artificial neural network began in the early 1940s. For half a century, it has experienced a long and tortuous road of rise, climax and depression, climax and steady development.
1943, psychologist W.S.Mcculloch and mathematical logician W.Pitts put forward the M-P model, which is the first model to describe the brain information processing process in mathematical language. Although the function of neurons is weak, it provides a basis for future research. 1949, psychologist D.O.Hebb put forward the hypothesis that synaptic connections are variable, and the learning rules proposed according to this hypothesis laid the foundation for the learning algorithm of neural networks. 1957, Rosenblatt, a computer scientist, put forward the famous perceptron model, which contains some principles of modern computers, is the first complete artificial neural network, and it is the first time to put neural network research into engineering realization. Because it can be used in pattern recognition, associative memory and so on, hundreds of laboratories invested in this research at that time. The US military even thinks that the neural network project should be more important than the "atomic bomb project" and has given huge funding, and has achieved certain results in the fields of sonar signal recognition. In 1960, B.Windrow and E.Hoff proposed adaptive linear elements, which can be used for adaptive filtering, prediction and pattern recognition. At this point, the research work of artificial neural network has entered the first climax.
1969, the famous American artificial intelligence scholars M.Minsky and S.Papert wrote the book Perceptron, which had a great influence. It was proved theoretically that the single-layer perceptron was limited, for example, it could not solve the XOR problem. They speculated that the perceptron capability of the multi-layer network was so much. Their analysis is like a pot of cold water. Many scholars feel that the future is bleak and they have changed careers, and the laboratories that once participated in the research have also withdrawn. After that, it was nearly 65438. During this period, Finnish scholar T.Kohonen put forward the theory of self-organization mapping, which reflected the self-organization characteristics, memory methods and the rules of nerve cell excitement and stimulation. Adaptive vibration theory (art); By American scholar S.A. Grosberg; Japanese scholar K.Fukushima put forward the cognitive machine model; ShunIchimari devoted himself to the study of mathematical theory of neural network, which has an important influence on the development of neural network in the future.
American biophysicist J.J.Hopfield published two articles in the Proceedings of the National Academy of Sciences 1982 and 1984, which strongly promoted the study of neural networks and caused another upsurge in the study of neural networks. In 1982, he proposed a new neural network model-Hopfield network model. In the research of this network model, he introduced the concept of network energy function for the first time, and gave the judgment basis of network stability. In 1984, he proposed an electronic circuit realized by network model, which pointed out the direction for the engineering realization of neural network. His research results opened up a new way for neural network to optimize associative memory and laid the foundation for the research of neural computer. Hinton et al. introduced simulated annealing algorithm into neural network in 1984, and proposed Boltzmann machine network model. BM network algorithm provides an effective method for neural network optimization calculation. In 1986, D.E.Rumelhart and J.LMcclelland put forward error back propagation, which has become a network learning method with great influence. 1987, R.Hecht—Nielsen, an American neurocomputer expert, proposed the back propagation neural network, which has the advantages of flexible classification and concise algorithm, and can be used in the fields of pattern classification, function approximation, statistical analysis and data compression. L.Ochua and others put forward a cellular neural network model in 1988, which has been widely used in primary visual processing.
In order to adapt to the development of artificial neural network, 1987 established the International Neural Network Society and decided to hold international neural network academic conferences on a regular basis. 1988 65438+ October neural network established. Neural Network 1990 IEEE Transactions published in March. China held the first academic conference on neural networks in February 1990, and decided to hold it once a year. 199 1, China Neural Network Society was founded in Nanjing. IJCNN92, jointly organized by IEEE and INNS, has been held in Beijing. All these have promoted the research and development of neural networks, and artificial neural networks have entered a period of stable development.
In the early 1990s, Edelman, a Nobel Prize winner, put forward a Darwinian model and established the theory of neural network system. In the same year, on the basis of predecessors' deduction and experiments, Xiang Yuan and others gave a chaotic neuron model, which has become a classic chaotic neural network model and can be used for associative memory. Wunsch put forward AnnualMeeting at the 90OSA annual meeting, which uses photoelectricity to execute ART. The learning process has the functions of adaptive filtering and reasoning, and has the characteristics of fast and stable learning. 199 1 year, hertz discusses the theory of neural computing, which is of great significance to the analysis of computational complexity of neural networks. Inoue et al. proposed using coupled chaotic oscillators as neurons to construct chaotic neural network model, which pointed out the direction for its broad application prospect. In 1992, Holland proposed a genetic algorithm to solve complex optimization problems by simulating biological evolution. In 1993, Fang Jian 'an and others used genetic algorithm to study the neural network controller and achieved some results. 1994, Angeline et al. put forward an evolutionary algorithm to build a feedback neural network based on the previous evolutionary strategy theory, and successfully applied it to pattern recognition, automatic control and so on. Liao established a new mathematical theory and method for cellular neural network, and achieved a series of results. HayashlY put forward an oscillating neural network according to the oscillation phenomenon in animal brain. 1995 Mitra combines artificial neural network with fuzzy logic theory, biological cell theory and probability theory, and puts forward fuzzy neural network, which has made a breakthrough in the research of neural network. Jenkins and others studied the optical neural network, and established a two-dimensional parallel interconnection and electronic optical neural network, which can avoid the network falling into local minima and finally reach or approach the ideal solution. SoleRV et al. put forward fluid neural network, which is used to study insect society and robot collective immune system and inspire people to analyze social large-scale system with chaos theory. 1996, ShuaiJW' and others simulated the self-development behavior of the human brain, and proposed a self-development neural network based on the discussion of chaotic neural networks. In 1997 and 1998, Dong Cong established and improved the generalized genetic algorithm, which solved the simplest topology construction problem and the global optimal approximation problem of the multilayer forward network.
With the development of theoretical work, the application research of neural network has also made breakthrough progress, involving a wide range of technical fields, including computer vision, language recognition, understanding and synthesis, optimization calculation, intelligent control and complex system analysis, pattern recognition, neural computer development, knowledge reasoning expert system and artificial intelligence. The subjects involved are neurophysiology, cognitive science, mathematical science, psychology, information science, computer science, microelectronics, optics, dynamics, bioelectronics and so on. The United States, Japan and other countries have also made remarkable achievements in the hardware and software development of neural network computers, and gradually formed products. In the United States, the neurocomputer industry has received strong support from the military. The Advanced Research Projects Agency of the Ministry of National Defense believes that "neural network is the only hope to solve machine intelligence", and only an eight-year neurocomputer project has invested 400 million US dollars. In the ESPRIT project in Europe, there is a special project: "Application of Neural Network in European Industry", and the production of neural network-specific chips alone costs 22 million US dollars. According to American data, Japan's investment in neural network research is about four times that of the United States. China is not far behind. Since the approval of 1990 Nankai University Optical Neurocomputer and other three projects, the National Natural Science Foundation and the National Defense Pre-research Fund have also provided financial support for the research of neural networks. In addition, many internationally renowned companies have also participated in the research of neural networks, such as Intel, IBM, Siemens and HNC. Neurocomputer products began to go to the commercial stage and were selected by national defense, enterprises and scientific research departments. In the world-famous Gulf War, the US Air Force used neural networks for decision-making and control. Under this kind of stimulation and demand, artificial neural network will have a new breakthrough and usher in another climax. Since the birth of the first neural network in 1958, its theoretical and applied achievements are numerous. Artificial neural network is a rapidly developing new discipline, with new models, new theories and new application results emerging one after another.
3 the development prospect of artificial neural network
In view of the problems existing in neural networks and social needs, the main development direction in the future can be divided into two aspects: theoretical research and applied research.
(1) Use neurophysiology and cognitive science to study the mechanism and calculation theory of brain thinking and intelligence, and study the theory by problems.
Artificial neural network provides a reasonable way to reveal intelligence and understand the working mode of human brain. However, due to the limited understanding of the nervous system at first, the understanding of the human brain structure and its active mechanism is still superficial and has some kind of "apriori". For example, Boltzmann machine has its advantages by introducing random disturbances to avoid local minima. However, it lacks the necessary brain physiological basis. Undoubtedly, the improvement and development of artificial neural network should be combined with the research of neuroscience. Moreover, some important problems raised by neuroscience, psychology and cognitive science are new challenges to the research of neural network theory, and the solution of these problems is helpful to the perfection and development of neural network theory. Therefore, using neurophysiology and cognitive science to study the mechanism of brain thinking and intelligence will change the understanding of the relationship between intelligence and machine if there is a new breakthrough.
Using the research results of the basic theory of neuroscience, this paper explores the artificial neural network model with higher intelligence level through mathematical methods, and deeply studies the algorithms and performance of neural computing, evolutionary computing, stability, convergence, computational complexity, fault tolerance and robustness, and develops new network mathematics theory. Because of the nonlinearity of neural network, the study of nonlinear problems is the biggest driving force for the development of neural network theory. Especially since people discovered the existence of chaos in the brain, it has become a new topic for people to use chaos dynamics to inspire the study of neural networks, or to use neural networks to produce chaos, because this is the fundamental means to study neural networks from the perspective of physiological essence.
(2) Research on software simulation and hardware implementation of neural network and application of neural network in various scientific and technological fields.
Because artificial neural networks can be simulated by traditional computers, can also be composed of integrated circuit chips, and can even be realized by optical and biochip methods, there is great potential to develop electronic neural network computers with pure software simulation, virtual simulation and full hardware implementation. How to combine neural network computer with traditional computer and artificial intelligence technology is also a frontier topic. How to make the functions of neural network computers intelligent and develop intelligent computers with functions similar to those of human brain, such as optical neural computers and molecular neural computers, will have a very attractive prospect.
4 philosophy
(1) Artificial neural network opens a new field of epistemology.
The problem of cognition and brain has long been concerned by people, because it is not only a psychological problem related to people's psychology and consciousness, but also a brain science and thinking science problem related to people's thinking activity mechanism, which is directly related to the answers to basic philosophical questions of matter and consciousness. The development of artificial neural network enables us to further understand the relationship between cognition and brain materialistically and opens up a new field of epistemology. The human brain is a complex parallel system with advanced brain functions such as cognition, consciousness and emotion. Artificial simulation helps to deepen the understanding of thinking and intelligence, and greatly promotes the study of the nature of cognition and intelligence. In the study of the overall function and complexity of the brain, artificial neural network has brought new enlightenment to people. Because there is chaos in the human brain, chaos can be used to understand some irregular activities in the brain, so the chaotic dynamic model can be used as a tool to model the external world and describe the information processing process of the human brain. Chaos and intelligence are related, and introducing chaos into neural network is helpful to reveal the mystery of human thinking in images. The key to the revival of artificial neural network is that it reflects the nonlinearity of things, grasps the essence of the objective world, and to some extent positively answers the most critical question of how intelligent systems learn independently from the environment. From the cognitive point of view, the so-called learning is the discovery and induction of unknown phenomena or laws. Because of its high parallelism, high nonlinear global function, good fault tolerance and associative memory function, strong adaptive and self-learning function, neural network has become a reasonable way to reveal intelligence and understand the working mode of human brain. However, due to the complexity of cognitive problems, at present we have no idea about the operation of neural networks and the internal processing mechanism of nerve cells, such as how information is transmitted, stored and processed in the human brain. How are memory, association and judgment formed? Does the brain have an operating system? At present, there is not much understanding, so it is necessary to deepen people's understanding of brain information processing mechanism in order to make artificial neural networks imitate all aspects of human brain functions.
(2) The driving force for the development of artificial neural network comes from the interaction of practice, theory and problems.
With the continuous expansion of people's social practice scope and the deepening of social practice level, the natural phenomena that people are exposed to are more and more colorful and complex, which urges people to explain different kinds of natural phenomena with different reasons. When different kinds of natural phenomena can be explained by the same reason, different disciplines cross-synthesize, and artificial neural networks are thus produced. In the initial stage, because these theoretical network models are relatively simple, there are still many problems, and these models have hardly been tested in practice, so the development of neural networks is relatively slow. With the deepening of theoretical research, the problems are gradually solved, especially after the engineering realization, such as the success of sonar recognition, before the first development climax of neural network is ushered in. However, Minisky thinks that perceptron can't solve the XOR problem, so can multilayer perceptron, and the research of neural network has entered a trough, mainly because the nonlinear problem has not been solved. With the continuous enrichment of theory and deepening of practice, it has been proved that Minisky's pessimistic argument is wrong. In today's highly developed science and technology, it is gradually revealed that nonlinear problems are the essence of the objective world. The interaction of problems, theory and practice ushered in the second climax of artificial neural network. At present, the problem of artificial neural network is low intelligence level, and there are other theoretical and implementation problems, which force people to carry out theoretical research and practice continuously and promote the continuous development of neural network. In a word, the previous reasons have encountered new phenomena with different explanations, prompting people to put forward more common and accurate reasons to explain them. Theory is the foundation and practice is the driving force. However, the role of theory and practice alone can not promote the development of artificial neural networks. It is necessary to ask questions to attract scientists into specific research areas and guide them to engage in related research, so as to approach scientific discovery. Then practice puts forward new questions, and new questions lead to new thinking, prompting scientists to keep thinking and constantly improve their theories. The development of artificial neural network embodies the dialectical unity of problem, theory and practice.
(3) Another driving force for the development of artificial neural network comes from the contribution of related disciplines and the competition and cooperation of experts from different disciplines.
Artificial neural network itself is a frontier discipline, and its development has a broader scientific background, which is the comprehensive product of many scientific research achievements. Weiner, the founder of cybernetics, studied human brain neurons in his masterpiece Cybernetics. Computer scientist Turing put forward the idea of B-net. I.llyaPrigogine put forward the self-organization theory of unbalanced system and won the Nobel Prize. Harken studied the macro-effect caused by the interaction of a large number of elements. The proposal and research of the "chaotic" state of nonlinear systems are all about how to establish complex systems through the interaction between elements, similar to the self-organization behavior of biological systems. The progress of brain science and neuroscience is quickly reflected in the research of artificial neural network, such as the theory of biological neural network, the principle of lateral inhibition found in vision and the concept of receptive field, which has played an important role in promoting the development of neural network. Hundreds of artificial neural network models have been put forward, involving many disciplines, which are dizzying and the wide application fields are amazing. Experts from different disciplines compete in different degrees in order to reach the leading level in this field, which strongly promotes the development of artificial neural networks. The human brain is an information system with very powerful functions and extremely complicated structure. With the development of information theory, cybernetics, life science and computer science, people are more and more amazed at the magic of the brain. At least so far, the signal processing mechanism of human brain is still a black box for human beings. To reveal the mystery of the human brain requires the joint efforts of neurologists, psychologists, computer scientists, microelectronics, mathematicians and other experts. In addition, philosophers should also be involved, and through the deep combination of philosophy and natural science, new methods to explore the nature and laws of human thinking will be gradually bred, so that the science of thinking will move from obscurity to rationality. Moreover, the competition and coordination of experts in different fields is conducive to sorting out problems and seeking the best solution. Looking at the development history of neural network, without the contribution of related disciplines and the competition and cooperation of experts from different disciplines, neural network would not be today. Of course, the application research of artificial neural network in various disciplines has in turn promoted the development of other disciplines and promoted its own perfection and development.