The Case of
The Frog's Eye
|describes the specialized way in which the frog's eye acts as a specialized interface for its own particular --- frogly --- purposes. Vision involves models but these models are usually so effective that they are never noticed until we understand the context in which they are defined.|
Nothing could be simpler, it might seem, than the act of seeing. We look and we know. I glance around the room and see chairs, tables, shelves, and books. Sometimes, I need to look more closely or to think about what some strange shaped widget might be, but it seems that I still know what I'm seeing even if I don't know "what what I'm seeing is". However, this simplicity is mostly illusion.
Vision, like other interfaces between minds and world, involves models. With vision, the models are so good that we are normally never aware of them. Apparent transparency is one of the hallmarks of a good model, because if you have to think about a model it means that something is wrong with the model in a particular context (but that something is possibly "right" with your more general understanding).
Vision's incompletness goes beyond its obvious limitations of resolution and point of view: we cannot see behind ourselves or at great distances (except with special optics) or above or below certain wavelengths (except with special gadgets). The models involved in vision are more than mere selections of certain wavelengths and points or ranges of view. To understand this, we need to look at the mechanisms of vision in the eye and brain, where patterns of focussed light striking the retina leads to responses to the objects which are emitting or reflecting the light.
A detailed description of vision in even the simplest animals would take many volumes and still be incomplete (there are many things we do not know). However, what we do know is that the sensors and connections of our visual system perform complex interpretations, reductions, and expansions on the signals they receive and transmit. Even as nervous signals leave the eye, they are both more and less than echoes of the light which struck the retina to induce them.
The modern mind is tempted to think of the eye like a television camera, breaking the patterns of color projected onto the retina into pieces and passing these pieces --- in some particular order --- along wires to the brain. But this is not the case. Even in the retina itself, light-sensitive cells respond in complex ways to the light that arrives. For instance, the receptors of the retina respond only to changes in light, rather than light itself. A motionless human eye is blind unless something moves in its visual field. Real eyes are always moving in tiny jerks, called saccades, which allow them to see things which are not themselves moving.
Signals --- news of changes --- leave the retina along chains of neurons which are not simple wires --- like the cable from your VCR to your television --- but are nerve cells which process and compare the information they are receiving. By the time the signal from a single cell has gotten very far, it has been combined and compared with signals from many neighboring cells which have likewise been combined and compared with it. Teasing out the function of these complex systems is much harder than finding a needle in a haystack: it is more like following a single strand of spaghetti in a jumbled and partially agglutinated dish of pasta.
Some of the most important work in understanding vision has been based on the visual systems of frogs. Part of the reason for this is that the retina of the frog has a uniform resolution and is more like a television camera than the eyes of other creatures. The eyes of mammals (like us humans) have an area of higher resolution (called the fovea) which are used for closer examination of objects in the world. When we "look at something" by turning our eyes, we bring its reflected light into an area of our eye with higher resolution.
The frog, however, has no fovea and it only moves its eyes to compensate for its own motion. This makes it easier to compare the patterns of light reaching the frog's retina with the signals leaving along its optic nerve. In early experiments, scientists placed electrodes along the nerve fibers emanating from the eye. Then, they shone a light on to the retina and looked at the pattern of responses along the outgoing nerves.
As we mentioned above, the optic nerve is less like a wire and more like a series of linked computers. Scientists placed electrodes at the outputs from the the last of these linked cells (called the "ganglion" cells). From these output fibers, they measured the signals produced in response to different patterns of light on the retina. Since the signals mingled with each other along the way, a given fiber always responded to an area of the retina rather than to an individual receptor cell.
The first studies of the responses in the frog's visual system were conducted in the late 1930's by the physiologist Hartline. Hartline introduced the term "receptive field" to describe the region of the retina to which an individual output responded. Hartline noticed that there were three very different kinds of reponses among the output cells:
Hartline also noticed that parts of some cell's receptive fields were more signifcant than others. Every cell had a certain region to which it always responded but certain cells had a surrounding area (which Hartline called the annulus) where changes could affect a response initiated by a change in the center.
Some numbers. The frogs retina contains about a million receptor cells and the about half-a-million output (ganglion) cells. The connection of input to output involves 2-1/2 to 3 million cells. These numbers present a certain mystery, as the whole assembly seems to be losing information. It is as though a high definition color television camera were being fed into fuzzy black-and-white monitor whose individual pixels flickered as things moved or changed. What is the darn thing doing and why are we using such an expensive camera?
In the early 1950s, the physiologist Barlow demonstrated the first clues which might resolve this mystery. He noted that the role of the annulus differed between the ON-OFF cells and OFF cells in the optic nerve. OFF cells fired more intensely if their annulus also darkened, while ON-OFF cells fired more when their annulus either darkened or lightened in contrast to the center of the receptive field. ON-OFF cells, it appeared, didn't only respond to differences in light over time (ON and OFF) but also responded to differences in light over space: edges. What might have first looked like noise turned out to have meaning: there might be an edge here.
The idea that the optic nerve might be recognizing patterns from the surrounding world was difficult to test, since the laboratory where scientists could measure nerve signals was a far cry from the frog's natural environment. However, in the late 1950s, Jerome Lettivin and his colleagues at MIT combined some new electrical measurement techniques with a certain kind of "frog virtual reality" to further unravel the mystery. In their paper "What the Frog's Eye Tells the Frog's Brain", they deciphered part of the function of the optic nerve of the frog and changed the way we look at seeing.
Their experimental setup partially surrounded the frog (and its eyes) with a dull metallic sphere. Objects placed on the inside of the sphere could be moved about by magnets and incredibly fine electrodes would measure the responses of the optic nerve (again, the output of the ganglion cells) to these motions. With this experiment, Lettvin and his colleagues reorganized Hartline's three kinds of ganglion cells (or their output fibers) into four types:
The first three of these fibers corresponded to the ON, ON-OFF, and OFF cells identified by Hartline and partially explained by Barlow. The fourth fiber, however, was something new and compelling. Informally, Lettvin and his colleagues referred to it as a "bug detector" since it responded to small moving dots in the frog's visual field which --- in the natural world --- would correspond to the insects on which the frog normally dined. The frog's eye didn't simply convey patterns of light and dark, but interpreted those images in a fashion which fit the frog's purposes in its natural environment.
Some simplifications. Though the frog's eye and visual system is much simpler than our human eyes, the description here has simplified it even more. There are other differences in kinds of fibers (some carry messages much faster than others) and the processing described here is only the first of many stages in the frog's visual system. Nonetheless, we can draw some important lessons.
The frog's eye is an interface which serves purposes and so acts as a model. Its purposes involve the survival, feeding, and reproduction of the frog. It ignores certain aspects of what it reacts to (the exact retinal image) while highlighting other aspects (the moving dots which might be insects). Like the model of years in computers subject to the "millenium bug," it is limited. It can be fooled by moving bits of metal in an experimental apparatus or even (in the natural world) a single grain of sand flashing across its view. However, the things which might fool it --- intrusive experimenters or single grains of sand --- are relatively rare in its world, making its assumptions relatively safe.
Looking at the other kinds of fibers, we see other purposes: shores and obstacles (sustained contrasts), intended or accidental motion (moving edges), or the presence of possible predators arriving from above (net dimming). In the last two cases, the motion which triggers the fibers could be either motion in the world (a moving object or predator) or motion of the frog itself (caused by a hop with its legs or the rocking of its lily pad). And this is only the beginning of the explanations we might construct and struggle to validate experimentally.
In addition to telling us something about models in vision, the story of the frog's eye also tells us something about how we can look at and understand models.
Hartline's original account of the frog's eye was based on measuring the response of nerve fibers to light shined on the retina. From these experiments, it was clear that the eye was doing something more than just passing signal from environment to brain. However, the real nature of the interface could not be understood until it was looked at in context. Barlow began this process by looking at patterns over regions of the retina --- center and annulus --- rather impulses at particular points. Lettvin and his colleagues extended this by immersing the frog in a "virtual reality" where the correspondence of output to environment could be more closely studied.
Models are invisible until we see the context of their purposes and environment. Because of this, we can only understand models by using other models which can describe their purposes and environments. All through this book, we will be looking at models by holding them against other models to reveal their context, their assumptions, their strengths, and their weaknesses.
We now turn from the eye of the frog to the memory of human children for time. Here we will see, again, how context and purpose can explain how a model's imperfections and failures may reflect a deeper, valuable, sensibility.