Chapter 4

The Universal Reinforcer

 

The Molecular Basis of Value

The modern revolutions in psychology, from Freudian and behavioristic psychology to evolutionary psychology have all been cursed by their troublesome implications, a dearth of useful procedures, and the difficulty in demonstrating unequivocally the validity of their theoretical components. This of course is not unexpected, since they were not complete theories in the first place, missing as they did the defining and refining power of a complete explanation that incorporated the true facts of how our brains actually work. But the inadequacies of the different schools of thought in psychology are shared across nearly the whole of academic psychology. Indeed, it is easy to take pause in near despair at the mounting pile of conflicting and complex data languages, theories, and facts that pile up in college libraries like an Everest of intellectual rubble.

Of course, a pile of rubble is utterly dependent upon your vantage point, or more specifically, your point of view. Focus in closely, and one can see castles (populated with ivory towers no doubt) rising amidst the debris. Indeed, psychology has made great strides in determining the neural substrate for the functional capabilities of the human mind. The mental functions of sight, smell, hunger, sexuality, memory, reinforcement, cognition and language can now be traced to specific neural processes. However, the conceptual integration of the functions that may denote the broad scale animation of the human body and mind and the purpose to which it is animated remains hobbled by other metaphorical conceptions that are as constraining to thought as the simple notions of old that mounted the universe on the back of a turtle. Indeed, to make these latter conceptions work as a broad based explanation for behavior and still cohere with outdated metaphors for the mind requires a crazy quilt of obtuse theorizing that makes even the complicated 12th century theories of an earth centric universe seem simple and elegant.

The upshot of this is that what we really want from psychology, or at least how the popular imagination wants to view psychology, cannot be served by a psychology that continues to base its arguments on incorrect conceptions of the basic processes underlying mind. Yet if the major metaphorical tenets that underlie 1st generation cognitive science and popular opinion can be replaced, then all of the pieces fall neatly into place, and with them ironically a validation of the core values that inform our lives and our cultures.

Modern science has shown that nature can be demonstrated to originate from simple and rudimentary first principles. The unification of the forces of nature and the subsequent reduction of nature to elemental structures distill nature to the logics of the calculus, evolutionary selection, or the biochemical helix of DNA. The full flower of nature, from blooming flowers to the flowering of the human intellect can be seen as the emergent property of the molecular principles that conceived our world and our individual lives. But the direction of nature, and in particular of human nature is not a fixed or a given thing. The future of our lives, and the life of the race, is indeterminate, and more harshly phrased, pointless.

The vision of humanity’s future as an exercise in pointlessness seems justified by the fact of the pointlessness of its present. The immense triviality and banality of popular culture certainly does not auger well for a future that will likely be populated by a routine workaday existence that is capped by an evening at home filled with endless variations and reruns of situation comedies, infomercials, and detective/medical/lawyer shows. Of course, even while Socrates was philosophizing with friends by some rock in Athens, the rest of the population was going about its business and pleasures, hardly needing to be troubled by the higher subtleties of life and its uncertain meaning.

And that is perhaps the point. The complex aspects of a civilization are invariably regarded as the acme of civilization. Yet, the exalted creations of art, music, literature, and science are hard to create and are often hard to understand. Indeed, complexity has a habit of being troublesome, and is easily shunted aside. But if troubles have a happy ending, then they can exalt cultures and individuals, and ironically match the genetically rooted impulses that create and direct minds.

 

The Hidden Language of Behavior

Minds live for complexity; otherwise we would all have no more gray matter or consciousness than slugs, tomatoes, or toaster ovens. We have to interminably figure things out, from languages both colloquial and mathematical to the common sense language of natural law. Our conundrum is that we all aim for mindlessness but know as well that we must be mindful of things. That is, we don’t want to be bothered but are restless until we are bothered. Our dilemma is how we can both lay back and still conquer the world.

Human beings have an instinct for problems, or more pointedly, finding problems. This instinct for craving change, or heterostasis, is in contrast to the more sensible or ‘homeostatic’ impulses that would rather have us lounge around in a hammock all day doing nothing more than performing the simplest chore. Whether we are inherently disposed to desire change or the resolution of change represents of course dual aspects of the same phenomenon, and the stop-go of challenge and restful resolution is characteristic of our daily lives.

Problems of course represent possible states of affairs that will likely happen given certain activity, or non-activity. Compared to our protozoan ancestors whose future could not be self-reckoned past a millisecond, as higher order creatures we can figure out things much further in advance. That requires of course an ability to model the future, which our mammalian brains kindly bestow, and an ability to attend to those aspects of the future that are worth thinking about.

Our early ancestor, a lemur like creature like it was, didn’t know what to think, or for that matter how to think. His brain in its then rudimentary state of development could not then handle language, but he nonetheless needed to model or look ‘forward’ to future events so that he could escape predators and find food. He didn’t have a formal symbolic language, and had to navigate complex worlds through an instinctive appraisal of shapes and sounds, and a perceptual mapping of the world that was shaped by experience. Keeping in mind a faraway goal, whether it was potential food or a potential mate presumably was a matter of neurological computation guided by the instinctive signposts of taste, smell, and sight. The role of emotion was rudimentary, and encompassed specialized and intermittent responses that enabled courtship, the stalking of prey, flight or fight, and little else.

Yet, in a complex world where there is so much to be aware of but only so few things we can be actually attentive to, a mechanism had to be in place to mark the value of the perceptual events that counted towards survival. Thus evolution provided our ancestors with sensitivity not only to sights, smells, and sounds, but also to the abstract elements of the environment that demanded attention and the investment of cognitive resources denoted by attention. As inheritors of this evolutionary baggage, we are not instinctively or natively drawn to good things per se, because the world is full of good things. Rather we are drawn to good things that require our attention. In other words, attention, as a by word for a conscious or non-conscious cognitive appraisal is aroused by situations whose import requires thought.

Until recently, the direction of attention was a mere matter of information processing that represented a perceptual order imprinted by experience that allows us to effortlessly navigate our world. Thus, as a benefit of experience, mice know which turn to take in a maze to get cheese, and humans know which way to turn to get to the supermarket. The importance of cheese and supermarkets was a mere function of information that was in turn guided by simple drives that acted as signs for the importance of goals or reinforcers. Although behavioral goals were guided by these instinctual or native sensibilities, the actual going was directed by information pure and simple. Thus the road map of experience was easily derived from the patterns of information or contingencies that we encounter each day. This behavioristic approach fits neatly into common sense, since it relegates the how of behavior to the outside structure of experience and neatly leaves out dealing with the inscrutable and complex mechanics of the brain. Behavior therefore was isomorphic to information derived from past (memory) and present experience. Underscoring this is the assumption that the brain is a mere computational device, and that the means whereby information is processed will not influence the native value of that information. Thus as thinking computers, we add and subtract and otherwise process information like some biological database, but the biological substrate like its silicon counterpart does not bias the result. In other words, the utility of our actions is based on information that for the most part is perceptual, and the very act of processing is value neutral. This seems on the face of it logical and sensible, but it is also false.

Unfortunately, as our earliest ancestors found out, to survive in a complex world required not just effective and rapid decision-making, but also a means to bias the mind to deciding upon, remembering, and persevering in those behaviors or skills that were most critical for survival. Thus, if a bear chases you, you must rapidly choose and continually enhance the right moves to escape him, persevere in your running even if he falls way behind you, and be able to immediately recall those moves so that they could be repeated if and when the bear caught his second wind. This biasing mechanism must speedily guide and enable choice, sustain the right choices, and sharpen one’s abilities to more effectively make future choices.

But before we may describe a biasing mechanism, we must first describe the perceptual events that set it in motion. Since the brain activity that instantiates choice reflects the moment to moment activation of arrays of neurons rather than the linear or serial processing of symbol like objects or electronic bits and bytes that can occur more leisurely in time, the perceptual events that instigate that activity must be on the whole simple, subtle, and be non or pre-consciously perceived. But what metaphor may be use to simply describe them? Ironically, the answer to this problem is a problem itself.

 

Discrepancy

Life is full of good things, but as many pundits periodically note, we tend to be scarcely aware of them. In bad times, a standing consolation is that at least we’ve got our health, our freedom, or our MTV. Yet we don’t normally revel in the fact that our kidneys function well today, or that a comet isn’t about to hit the earth any time soon. That there are so many things that are right and good in the world underscores the fact that we are not inherently disposed to pay attention to them, and for good reason. Survival just doesn’t depend upon them. Now this may seem at odds with a poetic view that takes delight in the lowliest things, yet survival also depends upon lowly things.

Whether beautiful or repellent, the ordinary facts of existence are the ignored facts of existence until we have to pay attention to them. In the large, we categorize the things and events we have to pay attention to under the rubric of work and leisure, and apportion our time according to attended events that we look or don’t look forward doing. Attention occurs when things are out of sorts, and the latter requires significant and often conscious information processing to solve or to resolve. Now out of sorts is not necessarily a negative thing, particularly when the solution is at hand or nearly so. In its simplest guise, being out of sorts is merely being new. An azure sunset, a comet in the night sky, a pretty turn of phrase or the smell of a new perfume gain our attention because they are new, and don’t quite fit into the sameness of our past experience. Now if a sunset, comet, or poetic metaphor has implications that span beyond the novelty of the sensory events they embody or invoke, then our attention has to stay a longer course, and our alertness rises or falls depending upon their scope and importance. Thus, if a comet were an auger for biblical events, we would surely attend to every meaning, both scriptural and scientific, that it entails. If the ‘problems’ attending a comet led to innocuous or benign results, then figuring out the meaning of comets will be a rather pleasurable journey of discovery. If on the other hand our mounting knowledge led to a prediction of an impending collision of the comet into the earth, then that same journey would be unpleasurable to say the least.

Now the solution to any large problem requires invariably the solution to a lengthy granular trail of small ones. As we evaluate the myriad options available to us to solve a problem, we are continually choosing between different response options that are reflected in the small scale or molecular perceptions that guide choice. These perceptions are guided by prior experience as encoded in memory and environmental changes as presently perceived. Moreover, some choices are much more critical than others, requiring a greater alertness to the fine grain details of the environment. Daily, we must not only make choices, but also weigh choices, and be able to nonconsciously ramp up the cognitive resources to make decisions effectively and to retain in memory the rules we used to make them.

So what is the simple basis or decision rule that our minds use to make the small decisions that in sum comprise our major choices? It is, in a word, a simple matter of discrepancy. A discrepancy is simply a molecular or small scale ‘problem’ that does not follow the common predictions that come from experience. We are indifferent to our environment unless we see something out of sorts, such as a bird flying backwards or blue trees. However, the ability to handle the oddity of backward birds or blue trees is dependent upon our ability to sort out innumerable choices. The multiple problems or discrepancies entailed by strange birds and trees requires us to rapidly shift attention between the many possibilities that can explain these perceptions. If the possible implications of these oddities were of little import, then they would unlikely direct our attention from our more workaday concerns. However, if their import was major (is their something in the water; or have aliens landed?), then attention would become fascination, and we would be very alert to every possibility. But the import, or the discrepancy or problem that import represents is ultimately not what gets our attention, but how we correct for the errors of our thinking which brought us to that problem to begin with.

 

Somatic Markers

A discrepancy implies an unforeseen error, or a mismatch between what was predicted and what actually happened. The correction of that error requires us to focus our attention, fix its object in memory, attribute a value to it so that we may anticipate it in the future, and correspondingly code all this in physical changes of the neural architecture or substrate that instantiates thinking and behavior. This entire process is what is called learning.

To change the way we know and respond to our world, or to ‘learn’ something requires the activity of biochemical agents that modulate or control neural activity, and here is the essential foundation of all our behavior, including our personal values, our emotional pleasures and pains, and consciousness itself. Learning means change, an alteration of sensori-motor associations and the overt and covert manifestations of those associations. Learned associations allow us to non-consciously predict our world, so we can put on our shoes, drive a car, compute numbers, etc. and still have the time to attend to other events (e.g. what to cook for dinner, what to watch on TV) that are not so predictable. The representation of this change is called reinforcement. A reinforcer occurs when a discrepancy and the error it designates is ‘corrected’, and thus can be more thoroughly predicted in the future. If the correction leads to ends that have positive import to an individual’s survival (e.g. food, sex, power) then the reinforcement is positive. If the correction leads to ends that have negative or no import to an individual’s survival, then we have a negative reinforcer.