Excerpts from my recent paper on semantic transparency - see previous post
Classical artificial intelligence (AI) holds that cognition is computational and so is primarily about information processing. Information is encoded within a cognitive system as symbols, manipulated according to rules to produce meaningful output. Symbols are persistent, reusable, and re-combinable elements that hold their meaning over time and across various contexts. The symbols are tokened in the system whenever they're needed to represent old concepts or to learn new ones. Systems that express these properties are called Physical Symbol System (PSS) architectures, because they run on physical hardware (whether biological or electro-mechanical) and rely on symbolic representation to model mental processes.
Symbols in PSS architectures carry meaning - express semantic properties - within the system. A system is said to be semantically transparent if the symbols express agreed-upon, natural language concepts. For example, in a semantically transparent PSS, the symbol "dog" is the symbol for a dog; "3" is the symbol for the quantity 3; and "John loves Mary" is an aggregation of symbols for "John," "loves," and "Mary." These symbols would be stored in the system's lookup tables or memory; the PSS architecture is structured so that these tokens are searchable and accessible. Simple or rudimentary ideas are thus represented by equally simple symbols, and the task of the system is to manipulate these, following the specified rules and requirements.
In semantically transparent AI systems, therefore, symbols closely map to the way humans articulate a concept. This highlights the significance of semantic transparency to researchers concerned with modeling human cognition. Clark (2001) notes, "These kinds of symbols reflect our own ideas about the task domain... they make it immediately obvious why the physical device is able to respect specific semantic regularities." Clark continues that since Classical AI holds that "intelligence resides at, or close to, the level of deliberative thought. This is... the theoretical motivation for the development of semantically transparent systems - ones that directly encode and exploit the kinds of information that a human agent might consciously access when trying to solve a problem."
Not all cognitive models rely on PSS architectures to represent information and carry meaning. In so-called connectionist systems, knowledge is not tokened in re-purposable symbols, as in the Classical AI model. Rather, information exists as a set of connection weights between the system's processing units. These units are connected to each other with various strengths, and each unit's activation (stimulatory or inhibitory) is a nonlinear function of the sum of all influences from the units feeding into it. Concepts are therefore expressed as a pattern of distributed activity; these models are often called parallel distributed processing (PDP) systems, acknowledging both the distributed nature of the activation patterns and its parallel (versus serial) processing methods.
Such connectionist networks have been found to be successful at both representation and learning. In practice, researchers feed a PDP system data and, depending on the output, tune the connection weights to reduce errors. Once suitably trained, a network can produce accurate output. A simple network was able, for example, to discriminate sonar echoes from undersea mines (vector output <1,0>) and rocks (<0,1>) (see Churchland, 1990). A more complex network was able to convert English text into recognizable speech, "discovering" the 26 English phonemes and the vowel/consonant distinction in the process (see Sejnowski and Rosenberg, 1987).
These systems cannot be said, therefore, to be "semantically transparent" like Classical AI systems. One cannot peer into them and find tokens representing natural language inputs or outputs, like "coffee" or "rain." Clark (2001) observes, "whereas basic physical symbol system approaches displayed a kind of semantic transparency such that familiar words and ideas were rendered as simple inner symbols, connectionist approaches introduced a much greater distance between daily talk and the contents manipulated by the computational system."
However, it is not impossible to regain the concept of "coffee" from a connectionist system. Smolensky (1991) notes that connectionism is committed to the notion that "mental processes are vectors partially specifying the state of a dynamical system (the activities of units in a connectionist network), and that mental processes are specified by the differential equations governing the evolution of that dynamical system." Since the system's knowledge inheres in a set of connection weights, it's possible to mathematically transform the output to reveal its constituent parts.
Clark (2001) observes, "The activation of a given unit... thus signals a semantic fact: but it may be a fact that defies easy description using the words and phrases of daily language. The semantic structure represented by a large pattern of unit activity may be very rich and subtle indeed, and minor differences in such patterns may mark equally subtle differences in contextual nuance."
Connectionist systems, then, are not without semantics; they are merely without readily (human-) accessible semantics. And what's more, the ability within the system to represent a range of values, not simply "on" or "off" states for each tokened value, as in Classical systems, means that connectionist systems offer more flexibility in representing ambiguities and, concomitantly, possibility or potential meaning in a way that a simple activated symbol system could never afford. For example, a connectionist system might read a series of sonar echoes and determine that the readings are more "mine-like" than "rock-like," but not necessarily one or the other, with, e.g., a unit activation pattern of < .8, .2 > rather than < 1, 0 >. Such system, in it capacity to represent these ambiguities, offers a broader expressive range than is possible with simple PSS architectures.
Churchland (1990) notes, "What we are confronting here is a possible conception of 'knowledge' or 'understanding' that owes nothing to the symbolic and sentential categories of current common sense and of traditional approaches in AI... An individual's overall theory-of-the-world, we might venture, is not a large collection or a long list of stored symbolic items. Rather, it is a specific point in that individual's synaptic weight space." In other words, even a system that does not represent knowledge with natural language may, nonetheless, be capable of inhering semantic content - content that is meaningful and, possibly even more representative of the continuous or "gray-scale" nature of human knowledge.
Sources and Further Reading
Churchland, Paul M. (1990). Cognitive Activity in Artificial Neural Networks. In Robert Cummins and Denise Dellarosa Cummins (eds.), Minds, Brains, and Computers: The Foundations of Cognitive Science. Malden MA: Blackwell Publishers, Inc., 2000; pp. 198-216.
Clark, Andy (2001). Mindware. New York, Oxford University Press.
Fodor, Jerry (1975). The Language of Thought: First Approximations. In Robert Cummins and Denise Dellarosa Cummins (eds.), Minds, Brains, and Computers: The Foundations of Cognitive Science. Malden MA: Blackwell Publishers, Inc., 2000; pp. 51-68.
Fodor, Jerry and Brian P. McLaughlin (1990). Connectionism and the Problem of Systematicity: Why Smolensky's Solution Doesn't Work. In Robert Cummins and Denise Dellarosa Cummins (eds.), Minds, Brains, and Computers: The Foundations of Cognitive Science. Malden MA: Blackwell Publishers, Inc., 2000; pp. 273-285.
Marr, D. (1982). Vision. In Robert Cummins and Denise Dellarosa Cummins (eds.), Minds, Brains, and Computers: The Foundations of Cognitive Science. Malden MA: Blackwell Publishers, Inc., 2000; pp. 69-83.
Sejnowski, Terrance J. and Charles R. Rosenberg (1987). Parallel Networks that Learn to Pronounce English Text. In Robert Cummins and Denise Dellarosa Cummins (eds.), Minds, Brains, and Computers: The Foundations of Cognitive Science. Malden MA: Blackwell Publishers, Inc., 2000; pp. 259-272.
Smolensky, Paul (1991). Connectionism, Constituency, and the Language of Thought. In Robert Cummins and Denise Dellarosa Cummins (eds.), Minds, Brains, and Computers: The Foundations of Cognitive Science. Malden MA: Blackwell Publishers, Inc., 2000; pp. 286-306.
