Welcome to my blog.  I want this blog to become a place where we can share information, good references, and ideas about how vision and perception interact to create our visual experiences.  

Sunday
May152011

Exploring Together

Great, you found this new blog.  Welcome. 

Vision, perception, and aesthetics -- I know this is pretty arcane stuff so why blog about it?   It started when I began to think about some of the basic guidelines that most designers consider when we do initial layouts.  These guidelines deal with things like the effective use of white space, evocative color palettes, appropriate typefaces, etc.   I'm aware of them and I usually respect them.  I know they work but I wish I understood more about why they work.  I want to know more about the processes by which someone senses my art with their eyes, perceives it with their brain, and ultimately has feelings about it?  

Believe me. I know these questions have sweeping breadth.  It is wildly optimistic to expect that all of them will be answered satisfactorily.  But let's start somewhere.  None of these questions are new; great thinkers have considered them for millennia.  One advantage we have today, however, is that there is a substantial new scientific understanding of human sight and visual perception.  There is even a renewed vitality in the field of aesthetics. 

Finally, I am not trying out for the part of professor here.  I have studied these topics enough to know how little I really know.   Let's explore this together.  I'll try to keep the ball rolling with a blend of posts and my best references.  There are many visual "parlor tricks" that actually reveal important clues about seeing and visual perception.  They also can be fun. 

In the next post, we will look at one of these "parlor tricks."  The results initially seem pretty benign.  We will see how these results demonstrate an important aspect of eye anatomy and reveal the brain's role in what seems like simple sight.  I’ll also share two of the best references I have found.  

Friday
May202011

What Does My Blind Spot Look Like?

Welcome back.  Let's do a simple visual test today, as promised.  The test may reveal some unexpected aspects of our own eyes.  For me, the test also raises new questions about how much, or how little, our eyes contribute to what we think of as vision.  

For the test, I would like you to go to the VPandA Images section of this site.  Before you do that, you may want to open a second browser window.  That will allow you to see the test pattern on that page AND still be able to read the instructions in this window.

On the VPandA Images page, please click on the image entitled Blind Spot Test Pattern.  It should enlarge on your screen.  If you are using a separate monitor or maybe something like an iMac, try taking the test directly from the screen.  If you have a laptop, I suggest that you right-click the image, save it, and print a copy for the test.

Either way, let's get started. 

1.  Cover your right eye. 

2.  Now get as close to the test pattern as possible, really close please, and stare at the topmost "+" sign.  Don't worry about focusing, that's not the point of the test; blurry works just fine. 

3.  SLOWLY, pull your head back from the test pattern.  You'll see the black dot and the horizontal line pattern but don't look directly at them; keep staring directly at the "+" sign. 

As you continue to pull back slowly, you will notice a point when the black dot disappears.  Stay focused on the "+" sign.  As long as you keep your head in about that same position, you will not see the black dot.  Part One of the test is over.  You seem to have a blind spot in the field of vision of your left eye.   Did you know that you had a blind spot?  It is perfectly normal.  

This blind spot is not an illusion; it is very real.  You may want to close the test pattern image now and open the image named "Drawing of Left Human Eye" and follow along.  

In the back of each eye there is a lining (the retina) that senses light, then it begins interpreting that information, and then it sends the information to the brain for further processing.  The retina is your only link to the visual world.  Without it, you are blind so it is hard to overstate its importance in seeing.  

The retina is the part of your eye that contains the "rods and cones," the individual sensors for detecting light.  Within the retina of each eye, there are about five million cones and 120 million rods.  Including both eyes, that is  a quarter billion sensors, each sending a critical stream of visual information to the brain.

In order for it to get out of the eye, the "bundle of wires" carrying the information must penetrate the eye lining.   The "bundle" is the optic nerve and it leaves each eye through an opening in the retina called the optic disc.  The optic disc is a small portion of the retina that has no rods or cones because that area is like a "grommet" through which the optic nerve travels.  Without rods or cones, any part of the image that falls on the optic disc, is completely lost.  The optic disc is a blind spot in each eye.  

Finally, the blind spot.  The image of the black dot is located in the upper left part of your screen.  The lens of your eye inverts the image projected onto the retina.  So the image of the black dot hits your retina (in your left eye) within the lower inside quadrant, the same quadrant that has the optic disc.  As you moved your head back in the test, the image of the dot moved along the back of your left eye wall.  When the image finally fell upon the optic disc, the black dot disappeared.

Did you notice the title of this post, what would a blind spot look like?  That seems odd because how could we ever see a blind spot?  So I don't know what a blind spot looks like.  

I do know that there is no explanation for how the eye, alone, could fill in the blind spot that we just saw with the surrounding white color.   The brain fills in the missing black dot using a mechanism that is still not completely understood.  

Here is another example with the same test pattern.  Again, move your head close to the pattern but focus your left eye on the LOWER "+" sign.  Pull your head back as before.  When the image of the break in the black bar lands on your optical disc, the bar fills in; there is no gap, just a solid black bar.

Again, we “see” things that the eye could not have seen.  We know there never was a solid bar; there was a clear gap.  Yet we “saw” the gap filled in. 

It is easy for me to forget my brain when I am using my eyes to see something.  All of the motor control is focused on my eyes while my attention is on what I am watching.  These simple experiments demonstrate the essential role of the brain in all that we see.   

Sunday
May222011

Say that again, please

Before we move on, I want to come back to some numbers I threw at you in my last post -- numbers that may warrant a closer look.  I told you that each human retina has about 125,000,000 "sensors" -- either rods or cones.  Did the magnitude of that really sink in?  

Just how big is a human retina?  A typical diameter for an adult human eyeball is about 25mm.  Of course it is not a perfect sphere but let's assume it is.  From various anatomical drawings, the portion of the eyeball lined with the retina starts at the back, of course, and seems to stop at about 70% of the back-to-front distance (the diameter, excluding the bulge of the cornea).  So if I did my arithmetic correctly, an adult human retina is almost 480 square millimeters.  (I ignored the optic disc area that is slightly less than 3 square millimeters.) 

What do these numbers mean in terms of common things?  The eyeball is about the diameter of a U.S. Quarter Dollar.  The area of one retina is about the same as the area of a regular U.S. postage stamp. 

OK, so we believe that there are 125,000,000 sensors within the area of a postage stamp?  That sounds very impressive but how does that compare to similar everyday items?  Let's compare the human retina to the "retinas" of two high-end digital cameras:  Canon's EOS 1Ds Mk3 ($7000) and Nikon's D3X ($8,000).   Both cameras have about the same number of sensors (between 22 and 25 million).  Because both cameras comply with the photographic standard for "Full Frame” images, the "retinas" for both have exactly the same surface area, about 860 square millimeters.  

Result:  Nikon- 28,000 sensors per square millimeter; Canon- 25,000 sensors per square millimeter; either of your eyes- 262,000 sensors per square millimeter.

And you received your two retinas for free.  Take good care of them!

Monday
May232011

Some Recommended Sources, Finally

I know, I know.  I promised several posts ago to begin listing some of my favorite sources related to this blog. 

My first recommendation is not a book at all but a website.  I don't say this often but this site is magnificent and spot on the topic.  When faced with stunning creativity, have you ever asked, "Why didn't I think of that?"  Candidly, after I found this site, it took me a few days to decide that it was still worth doing the blog you are now reading. 

The site is called the Mind Lab (http://jvsc.jst.go.jp/find/mindlab/english/base.html), produced by the Virtual Science Center of the Japanese Science and Technology Agency.  There are sixteen brief modules arranged in four groups:  1) Illusion of an uninterrupted world, 2) Constructing a 3D world from 2D images, 3) Visual interpretation of the physical world, and 4) Perception beyond sensory input.   The last four modules 13-16 get increasingly edgy culminating in the suggestion that "the causal relationship of our will and actions resulting from our will may be just another illusion." 

This site seems like Japan's gift to the world's curious kids, of all ages.

My only criticism is the music.  Initially intriguing, I soon found it distracting and then just annoying.  The SOUND-OFF control is at the bottom left of the screen.

Tuesday
May242011

Two Simple Administrative Items for Today

1) If you want to contact me privately on any matter, please use the Contact Gerber Grafix form.  Even though it may sound like there are many "corporate" readers, it is just me.   I would love to hear from you on any matter.  Praise is always appreciated but I value criticism and suggestions even more. 

2) Tomorrow I will start plugging some of the best books I have found on the subject of visual perception and aesthetics.  Before I do, we need some truth in advertising.  "I am a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.com."  That is exactly what my agreement with Amazon requires me to say.  Let me also say that I believe this arrangement brings benefit to you as well as some small compensation for me. 

Whenever I recommend a book, there will be a small Amazon widget next to my recommendation.  The widget will display details from the Amazon site.  Click on the widget and you go directly to the Amazon page for that book.  That gives you quick and easy access to other reviews of the book.  It does not obligate you to buy anything. 

I benefit from this arrangement in two ways:  Amazon automatically generates the widget code so it is a simple matter to cut and paste.  Whenever you use the widget and then purchase anything on the Amazon site, I receive a small monetary compensation. 

That's all I have for today.  As always, I would love to hear your feedback and suggestions.

Wednesday
May252011

I'd like to discuss two books today.  They both straddle the important connection between what our eyes sense and what our brains perceive.  

The first is "Vision and Art, the Biology of Seeing" by Margaret Livingstone.   There are other, more detailed and larger books, more akin to textbooks.   When I pick up a book on vision and art, however, it is a treat to see the material depicted in such a visually clear and appealing manner.  Dr. Livingstone's treatment of the role of luminance is particularly strong.  I have not found another book that treats this subject more effectively.  The book is beautifully produced with heavy glossy paper, vivid colors, and very readable content.  At just over 200 pages, it occupies a nice niche between a large textbook and small monographs.

The second book is "Eye, Brain, and Vision" by the Nobel Laureate, David H. Hubel. This older book (1988) is part of the Scientific American Library series. It seems to be out of print but there are used copies available in excellent condition at reasonable prices. I bought my copy through an Amazon reseller for $1. Typical of Scientific American publications, the writing is clear and the illustrations are first rate. The scope of this book is not unique but it has an interesting historical treatment of the evolving field and also some of the most important investigators.

Saturday
Jun182011

How Is Your Liver Today?

As we continue the shift from sight to visual perception, our attention must move to the brain.  Watch your step, please.  Most lay people, like myself, find the central role of the brain in visual perception to be almost counterintuitive.  

I exaggerate the role of my eyes in visual perception while forgetting the starring role of my brain.  That should not be surprising.  I am aware of my eyes throughout my waking hours.  I blink, I squint, and I seek better light or try to shade my eyes from too much light -- all in an effort to create the most meaningful personal visual experience.  My eyelids are even the “switch” by which I turn my vision "on" in the morning and "off" a night.

I usually think of my brain about as often as I think about my liver – almost never. They both operate faithfully without my awareness and they only signal me in the direst of circumstances.  Assuming that you have a healthy brain and liver, you probably know what I mean.   

How will we tackle the brain?   It might be interesting if we located key regions of the brain that are involved in visual perception along with the logic of key neurological connections.  Alternatively, we could skip all that and follow the brain's key tasks in visual perception.  I prefer the second approach.  We will examine some things more closely but the real payoff seems to be in "thinking like the brain" as it tries to construct a coherent 3-D awareness and understanding of the scene around us. 

Finally, here are three sources for valuable information. Two of them are entirely free. 

The first is a website http://viperlibnew.york.ac.uk/  operated by the University of York. You must register for this site but it is free.  There are a number of simple illustrations or short animations regarding visual perception.  The content is divided into 15 separate broad topics such as Anatomy and Physiology, Color, Depth, Illusions, Motion, and Abnormalities.  Most topics have several related presentations.  Many of the presentations are interactive Flash movies. 

One of the contributors to Viperlib, Tutis Vilus, teaches at the University of Western Ontario.  He has his own website as well where he posts excellent and absolutely free teaching materials about visual perception (Flash movies, PDF versions, and other useful information).  He actually offers these materials from two different courses:

The Physiology of the Senses     http://www.tutis.ca/Senses/

The Neurophysiology for Medicine    http://www.tutis.ca/NeuroMD/                                                  

Both sets make excellent use of Flash -- interactive and with sound.  The second set is a bit more advanced and oriented to clinical issues.  It still contains useful information for “the rest of us” and you can pick and choose as you wish.  My big point about all of these online teaching sources is that the interactive animations are great for learning dynamic topics like vision.   

The third source is a book: Vision Science – from Photons to Phenomenology, by Stephen E. Palmer.  This is my go-to favorite “textbook.”  The writing is semi-technical but very clear. Palmer does a terrific job presenting the material from multiple disciplines in layers.  There are even two tables of contents.  He also suggests a clear hierarchy of steps the brain uses to create a coherent visual perception of 3-D space.  This hierarchy is very useful in thinking about a field of such breathtaking complexity.  Palmer says it best: "Were it not for the fact that our brains manage to come up with the correct solution most of the time, it would be tempting to conclude that 3-D visual perception is simply impossible!" 

   

The Palmer book is large – over 2 kilos, 800 pages, and pricey -- $80.  If you want a definitive book on this field, start here.  If you get it from Amazon, of course, you have a 30-day return policy. 

 

 

Friday
Jun242011

The Raw Primal Sketch

To pick up from the last post, we are figuratively within the brain and we want to “think like the brain” about how it creates our visual perceptions.    We will skip the fine details of neural anatomy and physiology and stay at a highly conceptual level. 

The brain has access to an immense range of information, not only from the sensation of your eyes but also from monitoring the rest of your body and from what your brain has learned throughout your life.   The most important new information is, of course, the contemporaneous sensations of your two retinas.  

As I mentioned before, the form of that information from each retina is a series of “spike discharges” that remind me of a digital signal -- a series of 1's and 0's or in this case a spike and then no spike.  If you went to either of the two websites I recommended in my last post, you might have listened to sounds of these “spike discharges” from actual experiments in which certain light patterns were directed at photoreceptors in a retina. Those photoreceptor cells (the rods and cones) that provide information for a particular checkerboard square in your brain, first provide the information to other retina cells that aggregate the information before it leaves that retina.  There are about 125 million photoreceptors in each of your retinas but only about one million nerve fibers in the optic nerves that send that information to your brain.  

From my reading, as a non-scientist, of sources like Stephen Palmer’s Vision Science book, that I mentioned earlier, there seems to be a consensus among scientists regarding the gross steps by which your brain uses the information from your eyes to create visual perceptions. There are several big steps or tasks that seem sequential when we think about them.  More likely, each of these steps not only feeds the next step but also feeds information back to permit the revision of the previous steps.

Although these steps are reasonably well described in Vision Science and elsewhere, they are certainly not completely understood.

Your brain first receives information from your eyes, actually a separate set from each retina. (Remember that your eyes are located at different positions so the visual information from the left and right eyes will not be identical.) 

Imagine the organization of the information that your brain receives as a very large spreadsheet or perhaps a checkerboard.  The value within each square of the checkerboard represents a recorded measure of the spike "firing rates" from those photoreceptor cells within the retina that are associated with that particular square on the checkerboard.  These firing rates for each square in the checkerboard will vary in time according to the changing pattern of light that is striking the photoreceptors.   Think of information in each of the checkerboard squares as luminance values (shades of grey, from black to white) that come from those associated photoreceptors in that eye.  

As time passes, the overall pattern within the checkerboard changes in what vision scientists call an "optic flow."  Some of these changes result from the motion of objects within the scene.  Additionally, the motion of your head and particularly the motion of your eyes, even when your head is perfectly still, are other major sources of changes within the "checkerboard."  These latter eye movements are called saccades.  Saccades ensure that eyes' region of best visual acuity (a 20 degree cone centered on each eye) is always pointed towards the highest interest areas within the visual scene.  If you are startled by something in front of you, for instance, your eyes will shift to that location in less than a second.  "Routine" saccade movements, like those during reading, occur even more quickly and continuously.  Each one of these rapid involuntary eye movements results in an equally rapid change in the contents of the "checkerboard."  

Unlike a simple animation, the notion of an “optic flow” more resembles a movie of super high-speed photography, e.g. the classic, slow motion movie of, let's say, a bullet passing through a pane of glass.  If I did my arithmetic correctly, the implied data rate of this optic flow is in league with the capacity of the residential high-speed Internet connection to my home.

Upon getting this information from the eyes, your brain first constructs a so-called “Primal Sketch” of the scene from the available information.  To construct this Primal Sketch, your brain must search for clues about the objects that make up the scene. This earliest stage has been called a “raw Primal Sketch.”  Your brain tries to detect significant cases of sharp linear contrast in the pattern of luminance values within the “checkerboard” array of retinal data from the scene.  

One obvious example would be the detection of lines of sharp contrast that are probably edges of some object.  Another example of linear sharp contrast is a bar shape, i.e. a simple rectangle, that could be an open door within in the scene.  There are also shapes called “blobs” and a fourth category called “line terminations.”  

As I hope you saw for yourself in the videos I recommended in my last posting, these patterns are not imagined by scientists as what ought to be.  Each pattern is the empirical result of experiments to define that pattern for which a particular photoreceptor cell is tuned to react.  There is even one classic example on the site I recommended in which the Nobel laureate, David Hubel, draws the bar shape with a felt-tip pen right on the screen, based upon the Geiger-counter-style sounds he hears as he moves the light source around in the vicinity of the photoreceptor cell.  

One final thought.  The fact that your brain’s initial understanding of a scene requires a logical distinction between a line and the termination of a line seems to be a compelling demonstration of just how far your brain must go before it can give you a useful visual perception of a scene.  And remember, your brain does all this, and more, so quickly that you are unaware it is ongoing.  

So now we have the raw Primal Sketch.  Next time we will move on to the full Primal Sketch.

Sunday
Jun262011

Rubber Checkerboards

Just a few thoughts to share today; let's consider the usefulness of the idea of a matrix or checkerboard when we think about the relationship between a small region of the visual scene and a corresponding region of the retina.  You saw that I made extensive use of this checkerboard metaphor in my last post.  You will see it again in future posts.  

We know that the lens "projects" an image of the outside scene onto the retina.  If you did the blind spot test earlier in this blog, you learned that there really is a correspondence between objects in the real world -- like the black spot on the test sheet -- and specific locations on the retina.  Remember that you had to move your head in order to move the retinal image of the black spot until the image fell upon the "blind spot" in the eye you tested.  

In the early introductory portion of the Stephen Palmer Vision Science book, he portrays the organization of information from the retina in what I have called a "checkerboard" -- in his specific example it was a matrix with 24 rows and 17 columns.  He demonstrated, at least to my satisfaction, that this is ONE useful way to portray the spatial mapping of photoreceptor data to the portions of the brain that use that data.  Useful as it may be, he certainly does not suggest that a matrix or what I call a checkerboard is anything more than a metaphor.  

He later shows that the actual data would pose significant problems for this checkerboard metaphor.  The reason is pretty simple.  Consider the one important property of any checkerboard -- a matrix of evenly spaced rows and columns.   Although the rows and columns in a matrix or checkerboard are evenly distributed -- the distribution of the actual photoreceptors on the surface of the retina is certainly not even.  

The central foveal region of the retina has a very high density of photoreceptors within a 20-degree visual cone around the central line of sight of each eye.  In order to fit the information from this high-density region of photoreceptors into our evenly spaced checkerboard, we would need a “rubber checkerboard” that could bulge to permit us to pack in the additional information.  This phenomenon is called "cortical magnification."  It is called "cortical" because the magnification effect first matters when the information from the eyes arrives at the primary visual cortex in the very back of your skull.  [Oops, I slipped and mentioned a specific part of the brain.]  ;-)

The images below are my very rough attempt to visually depict "cortical magnification."   You probably recognize the left-hand image from the landing page of this website.  Imagine that you are standing where the camera was that produced the image of my "virtual world headquarters" lobby.  The image on the left is what you would see.  The image on the right is exactly the same view but I have tried to distort the image as if you had packed extra information from the high-density foveal region into a bulging rubber checkerboard.  The optical diameter of the bulge is about 20 degrees.

 

Tuesday
Jul122011

Watch Your Step. The Path Gets Steeper From Here.

Welcome back.  Let's resume our journey that began in the outside world of light and color, then continued through the retinas of our eyes, and on to the brain where sensations become the visual signals needed for perception of that outside world.

In his interesting book on the "Theories of Visual Perception," Ian Gordon describes three levels of theoretical development in any science:  1) Great theories, such as Newton's theory of gravity, or Einstein's theory of relativity.  Scientific theories evolve and mature through relentless experimental challenges.  Great theories are sufficiently refined to withstand foreseeable new experimental attacks.  We can count on great theories; the physicist Richard Feynman once called Newton's theory of gravity "the most powerful equation of all time."  2) Good theories lack the stature of great theories because predictions from good theories are not always reliable.  3) The remaining ideas in scientific fields like visual perception might be called Working theories -- "coherent sets of ideas that have prompted high quality scientific research." 

Referring to his own book about visual perception, Ian Gordon flatly states, "There are no great theories in this book."  He mentions the work of David Marr and the Young-Helmholtz theory of color perception as candidates to become good theories someday  (more on that later).  He concludes, "Given that there is no general agreement among vision researchers on what needs to be explained about perception -- conscious experience, neurophysiological mechanism, and so on -- it is not surprising that theories of visual perception have so far lacked the rigor and power of the great scientific theories.  We should not be depressed by this fact.  The brain is the most complex system in the know universe.  It may never be fully understood."   I personally think that is a bit harsh but I am not the expert.

As we move from the notion of a "raw" Primal Sketch of the outside world to the so-called "full" Primal Sketch and beyond, expect to experience substantially more uncertainty as well as complexity.  We seem to be in the midst of an animated refinement of working theories for visual perception.  We will encounter a much steeper terrain in this blog, made so by the inherent experimental challenges and also by what seems to be a substantial increase in the scope and complexity of the actual cognitive processes involved in visual perception. 

Forgive an admittedly simplistic analogy but the situation reminds me of how a group of people approaches a jigsaw puzzle.  They begin by dumping all of the pieces onto a suitable table.  Then they prepare the pieces for building the puzzle by flipping them so the pieces all face up.  The puzzle builders must do this in order to SEE all the pieces.  They next organize the pieces in the best manner to build the puzzle, maybe according to dominant colors, distinctive patterns, etc.  A few of the pieces have special significance -- those with straight edges and particularly the coveted four "corner pieces."  That stage seems to be relatively straightforward and rule based. Perhaps it resembles the first steps in visual perception, from the eyes through the "raw" Primal Sketch.

With all the puzzle pieces prepared and the special pieces identified, the builders are finally prepared to assemble the pieces into a mosaic that represents the SCENE portrayed on the puzzle box.  This is the much more difficult and complex task and, by analogy, so is our journey ahead.   

I think it is only prudent to remember the strengths as well as the limits in our understanding of visual perception. Just how mature is the science in this field?  As the old saying goes, “This is not rocket science.”  We can only wish it were, because rocket science is simple by comparison to visual perception.

If you are young and curious about neuroscience, Ian Gordon’s somber grading of visual perception theory should give you great hope.  In a world where so many important scientific mysteries have been solved, visual perception remains an exciting new frontier!