[AI@50]
Filene Auditorium, Moore Hall
Dartmouth College
Hanover, NH
Conference notes by Meg Houston Maker
Geoffrey Hinton and Simon Odinero, Toronto
From Pandemonium to Graphical Models and Back Again
Odinero gives this talk solo
How to build a perceptual system -- a "pandemonium architecture" after Selfridge
The good old fashioned neural network: input - hidden layer - output layer, plus a back-propagation algorithm as a derivitive of learning.
What's wrong with this model?
- you need a labeled training corpus
- it's not efficient; unless the weights are redundant, labels can't provide enough information
- the learning time doesn't scale well with more than 2 or 3 layers
- neurons need to transmit the signal forward and back-propagated information backward; real neurons don't do this
Overcoming These Limits -- using a Restricted Boltzmann Machine
A simple model is a 2-layer network (a shallow network): 1 layer of hidden units (with no connections between them) and 1 layer of visible units. Start with training data on the visible layer (images of reality), update the hidden layer, then update the visible units in parallel to get a "reconstruction" of reality. Then update the hidden units again. This network can learn to recognize features of the training data, but it tries to frame all perceptions in terms of that corpus (so if it were trained on the symbol 2, and was given a 3, it might recognize some features of 3 that correlate with those of the 2 (e.g., the top curve to the left).
It is also possible to create a more complex model that has multiple hidden layers. This seems more promising than back-propagation because there is only one type of signal sent through the system. This system can "fantasize" outputs that map to reality. So if you tell such a system that this symbol is a 2, it can create output shapes that look like 2. Likewise, if you give the system a hand-written 2, it can guess most of the time that it represents the numeral 2.
MHM editorial: There is no evidence that this system carries or expresses any understanding of the meaning of "two-ness."
Rick Granger, Dartmouth College
Essential Circuits of Cognition: The Brain's Basic Operations, Architecture, and Representations
Granger uses a Mac.
The Problem: Intelligence is undefined; it's only defined by example, by what we've learned. There's no formal spec. The only spec we have comes from brains. The question is how to scale from brain mechanisms to high level faculties. One approach is an analysis of brain circuits and systems, and a deriviation of alrorithms and data structures, not just statistics, and anatomically structured systems that construct high-level cognition. It's not a debate between statistics and algorithmic analyses. The issue is to show how logic arises from statistical methods.
As mammalian brain size increases, specializations decrease. In large brains, the posterior and anterior cortex are heavily connected, and the output of the anterior cortex, which in small mammals drives the musculature, has feedback mechanisms back into the anterior cortext. These feedback architectures are repeated redundantly in the brain in nested clusters and drives more complex behavior/output.
Granger shows an example diagram of a core loop within the brain (a thalamocortical circuit) that acts as a feed-forward network in recognizing, in his example, a flower. As the output of one region becomes input for another region, more complex patterns are created, generating a kind of grammar within the system. So, at the end of the day, there are a few algorithms that can be derived from brain circuits, and these are fodder for AI research.
For more, see BrainEngineering.org
Comments