From pixels to surfaces
The ingredients of reflected light
So far, we have been considering the spectral structure of points (pixels) in an image and how they are processed by the visual system. However, the visual system is often not directly concerned with such structure—instead, we are interested in how it relates to properties of the environment.
Hence, we need to think about how the spectral structure relates to the environment. The spectral structure contains the influence of both the lighting in the scene and how the visible surface reflects incoming light.
In the simulation below, you can vary the light source and the type of surface. Note that they both have values at each wavelength. For the light source, this is a measure if its intensity at that wavelength. For a surface, this is the proportion of light that it reflects at that wavelength.
The line marked 'Reflected' is the light that is reflected by the chosen surface under the chosen lighting. Note how it is the product, at each wavelength, of the lighting and the reflectance.
Colour constancy
Vision seems to be much more concerned about the surface reflectance property of the reflected light than either the lighting or the reflected light itself. Our ability to discern the reflectance of a surface across variations in the lighting is known as 'colour constancy'.
Colour constancy can be apparent in a variety of interesting 'illusions' (we will talk more about whether 'illusion' is a good name for these demonstrations). In the figure below (from Purves, Lotto, & Nundy, 2002), the tiles that appear blue in the left image and those that appear yellow in the right image have the same spectral distributions of reflected light. However, they tend to be perceived as different colours—with the 'illusory' colours corresponding to the surface reflectances that they would need to have to been under yellowish (left) and blueish (right) lighting to produce the spectrum of reflected light.
If you're not convinced that they are the same, hover your mouse over the image—the context will be masked and the relevant tiles will be shown in isolation. They should now appear to be very similar.
Lightness constancy
A similar phenomenon is apparent for achromatic surfaces—greyscale surfaces without colour. We use the term 'lightness' to indicate the perceptual impression of the reflectance of a greyscale surface. The term 'brightness' is used to refer to the perception of the combination of the surface reflectance and its illumination—the term for the associated physical quantity is 'luminance'.
The image below (The "Checkershadow illusion" by Ted Adelson) is a striking demonstration of "lightness constancy". In the demonstration, the panels marked A and B have the same luminance. Again, you can hover your mouse over the image—the context will be masked and the two checks will be shown in isolation. They should now appear to be very similar.
An artist's perspective on colour constancy
The video below ("Colour constancy illusions and painting" by David Briggs, a Sydney artist and teacher) is a great perspective on the topic of colour constancy.
Note how the video emphasises clarity of terminology. It also relates to why I (Damien) tend to avoid referring to these demonstrations as 'illusions'. They are only really 'illusions' if you think the goal is for perception to correspond to the signals present at the sensory receptors rather than the important causal features of the world.