Because of inhibition, the brain recognizes contrast in visual information, not the information itself. Simultaneous Lightness Contrast is the term meaning the appearance of something being affected by the appearance of things surrounding it. There are many popular examples of this demonstrating how the eye can easily be "tricked."
Because we can only see contrast, we rely almost entirely upon edges to see. This can easily be demonstrated by the following example:
What's happening? Neither the left nor the right side of this image is actually a single shade box - each is a gradient. The right side slowly gets darker as it gets closer to the center, and the left side slowly gets lighter. The change is so gradual that your eye does not recognize any edge so does not even notice a color change. In the middle there is a wide contrast, and the eye sees the edge. Thinking that there are not more edges, it "fills in" the left and right sides to the shades in the middle. When you put your finger over the line you remove the edge, so you only see one single-shaded box!
Lightness isn't the only contrast that the brain recognizes. When the brain analyzes visual signals, it notices many different aspects, called channels, of the image. Simultaneous contrast can be seen in all of the channels, just as it can be seen in lightness. In all of the below examples, place your mouse over the image to make the illusion disappear.
Other types of channels include the red, blue, and green color channels which are described in detail in Color Perception.