Socratic Wisdom, Judgment and the Uncertainty Principle

The Concept of Socratic Wisdom

One sign of intelligence is to "know what you know and to know what you don't know."; During his defense speech at his trial, Socrates said "The only thing I know is that I don't know anything." According to the Oracle at Delphi, Socrates was "the wisest of the Greeks." Socrates claimed that he was wiser only in knowing the limitations of his knowledge. Most machine learning systems lack Socratic wisdom. They lack introspection and, if they make an estimate of the probability that an answer they give is correct, they generally over estimate.

The Difference Between Knowledge and Wisdom

Knowledge and wisdom are distinct concepts. It is possible to know a lot of facts while lacking wisdom in the sense of Socrates. Computer systems are very capable of storing huge quantities of data with perfect recall, but that is only knowledge not wisdom. Modern AI systems, including deep neual networks, can perform many tasks as well or better than humans. These systems may satisfy the original definition of artificial intelligence, but it is not the same as wisdom. There are many aspects of human intelligence on which AI system still do very poorly. Some of these tasks are very easy for humans:

• Realizing when they don't know something
• Knowing when they are not sure
• Recognizing that a person or system has made a "stupid mistake"
• Recognizing that an image is not realistic

Introspection and Judgment

Introspection is a necessary but not sufficient condition for wisdom. If wisdom is the long term objective, then introspection is a stepping stone. However, even introspection may be too difficult as an immediate goal. For example, many highly intelligent people are poor at introspection. Introspection requires judging oneself. Many people are much better at judging others than judging themselves.

The Observer Effect and the Uncertainty Principle

However, the act of judging involves a phenomenon that social scientists call "the Observer Effect:"

    the act of observing a person or system changes the person or system being observed.

In quantum mechanics, there is an unavoidable observer effect call the "Uncertainty Principle." In social science and neural network training, the observer effect is not absolute and may be reduced. A judgment node is a new concept that breaks the normal rules for training a neural network in order to observe and judge a node or subnetwork while avoiding the direct consequences of the observer effect.

Judgment Nodes and Selective Back Propagation

A judgment node is special node that judges another node or subsystem and that is integrated into a larger system by a companion integration node that breaks the rules for back propagation for gradient descent. A judgment node provides judgment or even introspection within a single system while the companion integration node limits the observer effect.

Besides providing introspection, a judgment node has unique properties that are useful in many other ways:

• A judgment node can accelerate training
  • The weights of a companion integration node are pre-determined. They do not need to be trained.
  • It is often easier to train a judgment-integration pair than a comparable set of conventional nodes.
  • With judgment nodes, a deep neural network can be trained without back propagation.
• A cascade of judgment nodes can build a network of arbitrary depth.
• A judgment node can predict the relative performance of subnetworks in classifying a given datum.
• A judgment node can help build large ensembles with diversity.
• A judgment node can usually fix an individual error without causing new errors.
• Combined with knowledge sharing, judgment nodes help can build more interpretable networks.

Navigation Menu

by James K Baker and Bradley J Baker