“ Turing Award ” winner 、 Dean of the Institute of cross information, Tsinghua University 、 Yao Qizhi, academician of the Chinese Academy of Sciences 2000 year 10 month 22 In the speech at the Japan Pujiang Innovation Forum （ Reference reading Interpretation of Yao Qizhi's latest speech on the development of artificial intelligence !）, This paper clearly puts forward the system of artificial intelligence “ Interpretability ” The problem is a major bottleneck restricting the development of artificial intelligence . From the perspective of embedded system , On this issue, we will explore what interpretability is ？ The origin of interpretability and its impact on the future development of artificial intelligence ？
informally , Interpretability is the recognizability of things 、 The degree of understanding , So that people can understand things from “ What is it? ” Advance to “ Why? ” The state of . Artificial intelligence system “ Interpretability ” It is embodied in the hardware of the system 、 Knowledge and understanding of software and chip .
The problem of interpretability began in the era of artificial intelligence after the birth of intelligent tools . In the era of the industrial revolution when mechanized tools were born , There is no question of interpretability . In order to control mechanized tools effectively , Besides theoretical knowledge , You also have to be familiar with the structure of mechanized tools 、 Operation principle , For example, the driver should know about the engine 、 tank 、 Oil way 、 ignition system 、 Brake system , To drive a car safely and reliably ; Train drivers need to know about boilers 、 Steam pipes 、 steam engine 、 Brake mechanism , To control the train . In everyday life , For the sake of the common people, the electric appliance manufacturers can use their products effectively , Competing to provide users with the best product interpretation services .20 century 6、70 years , Radios for sale 、 recorder 、 Record player 、 TV sets are equipped with detailed circuit diagrams and instructions . Today's artificial intelligence era , Almost all of the intelligent products have become invisible to users 、 The black box that can't be understood , But users don't mind the product's “ No one knows ”. Product maintenance “ Replaceability ” Processing mode also does not need to understand the product structure and operation principle . The era of artificial intelligence seems to have entered an era in which the interpretability of systems can be ignored .
To understand the great difference between the era of the industrial revolution and the era of artificial intelligence in the interpretability of product systems , We can start from the differences between the products and industrial models of mechanized tools and intelligent tools .
In terms of product differences , First , In terms of Architecture , Mechanization tool is a product system of structure and function integration , Intelligent tools are a structure （ Hardware ） And function （ Software ） Separate product systems .20 century 5、60 Radio of the age 、 recorder 、 Record player 、 Electrical instruments , All functions depend on the circuit structure of various functions ; Today's smart appliances 、 mobile phone 、 The computer 、 e-book 、 Zhihuiping, etc , None of them is an electronic system with software and hardware separated , All the functions of the system are given by software , Hardware only provides a running platform for application software . This separation causes consumers to no longer care about the hardware structure of the product . secondly , Product hardware structure platform and continuous optimization of ecosystem , Make the products have high reliability , Even if there is a failure, it's mostly a software problem , The user takes a restart measure , The maintenance department can solve this problem by reloading the system software or directly replacing the control module , There is no need for more knowledge of hardware architecture . Third , From the mechanism of function realization , The functional mechanism of mechanized tools is “ people + Tools ”, Take the camera, for example , All the functions of the camera （ focusing 、 Aperture and shutter settings ） All have to be done by the user ; The function mechanism of intelligent tools is the independent operation mode of tools , When users use digital cameras , Just take a shot 、 Just press the shutter , There is no doubt that the user regards the digital camera as a black box that can be used to take pictures . It is the separation of product structure and function （ There is no direct relationship between structure and function ）、 The separation of software and hardware （ Hardware on stage, software on stage ）、 The separation of man from function （ People have nothing to do with the implementation of tool functions ） It directly leads to the unexplainable or unexplained of intelligent tools .
From the perspective of industrial model , The problem of interpretability is not only in product systems , It also exists widely in the industrial structure . The industrial structure in the era of industrial revolution is a closed structure of integration of innovation and application （ The enterprise innovates itself 、 Apply it yourself ）, There is no question of interpretability . The age of artificial intelligence , Scientific and technological innovation and the application of innovative achievements have been completely separated . Technological innovators specialize in technological innovation , And transform scientific and technological achievements into various types of knowledge platforms （ Chip platform 、 Hardware platform 、 software platform 、 Product platform, etc ） Not involved in the application of scientific and technological achievements ; The users of innovative achievements are not engaged in scientific and technological innovation , On the basis of the platform, the final innovative application of scientific and technological achievements is completed . Knowledge platform consists of unexplained knowledge principle and interpretable application interface , The figure below shows the fan-shaped ecosystem mediated by knowledge platform in the era of artificial intelligence .
The fan-shaped industrial ecosystem in the era of artificial intelligence
In this fan ecosystem , The knowledge platform divides the knowledge principle and application interface into two parts: unexplainable and interpretable . The unexplained principle of knowledge effectively protects the intellectual property rights of knowledge innovators ; The interpretability of application interface ensures the platform's fool application , It greatly reduces the knowledge threshold for the application of innovative achievements . Under the intermediary of knowledge platform , Knowledge innovation enterprises and many innovative knowledge application enterprises constitute an upstream and downstream fan-shaped industrial ecosystem . From the perspective of the evolution of social productivity , This is undoubtedly a very advanced global industrial ecosystem .
It can be seen that , The age of artificial intelligence , Whether it's a fool application of knowledge platform , Or the industrial division of labor in the ecosystem , All in “ Non interpretable ”、“ No need to explain. ” Normal operation in a normalized environment . So why did someone say , Interpretability has become a major bottleneck restricting the development of artificial intelligence , It has become a hot issue in the research of artificial intelligence , This also needs to find the answer from the analysis of the industrial ecosystem .
First , From the perspective of introducing social intelligence resources to the maximum extent , The incomprehensibility of knowledge achievements of knowledge platform is not conducive to gathering talents for knowledge innovation ; secondly , The incomprehensibility of knowledge platform strengthens industrial monopoly , It is easy for industry to become rigid ; Third , Incomprehensibility reduces the user's trust in the product , Also left many black box operation for the manufacturer （ Steal user information 、 Harm the interests of users 、 Control the product feature orientation 、 Intentional concealment of product defects, etc ） Space . therefore , People have been trying to change the unexplained situation of knowledge platform and knowledge achievement , Its main strategy is to open source widely .
The first open source is the hardware system of artificial intelligence , From the user board 、 White system to raspberry pie product platform , There are also development boards for user products 、 Emulator 、 Debugging tools , so far , The hardware system has a completely transparent application environment ; Open source software starts with the operating system , The big universal ones are 1983 year GNU The plan started Linux operating system , as well as Linux Many open source operating systems derived from kernel , Most of them are used in embedded systems RTOS It's all open source software , Today's Internet of things operating system also implements a wide range of open source mode ; Chip open source started late , But the momentum is strong , At present, the most important open source chip is RISC-V.
In Yao Qizhi's speech , When it comes to interpretable bottlenecks , The concerns about the interpretability of machine learning algorithms are highlighted . The interpretability of algorithms belongs to the field of open source software . Because the function of intelligent system is given by software , The non absolute reliability of software programming determines the bias and vulnerability of software algorithm , So as to form the reliability of artificial intelligence system 、 Credibility and security . The interpretability of the algorithm will seriously hinder the solution of these problems .
Artificial intelligence is not a simple industrial revolution , It's the knowledge revolution of modern computers , Its influence has spread to all fields related to human knowledge . The complete separation of knowledge innovation and innovative knowledge is an important feature of the times of artificial intelligence . After separation , In the intelligent tool industry and user use , Unexplainable or unnecessary to explain the principles of knowledge . The former eliminates concerns about intellectual property protection in the process of global industrial transfer , The latter forms a fool application mode of intelligent tools . However , This separation results in the incomprehensibility of platform knowledge principles, which is not conducive to the future development of artificial intelligence . A wide range of open source strategies in the field of artificial intelligence , Open source hardware 、 Software open source 、 Chip open source 、 Open source algorithm , It will eliminate the interpretability barrier , Make AI take off in an all-round way .