1. Fundamentals of Color Coding for Data Search and Grouping

Figures 1a and b. Which category has the fewest stars: 32-, 24- or 16-pointed stars? Click the figure to see full-screen details.

1.1. Color-Coded Visualization

Color coding has diverse applications to data visualization. A general principle is to align salience of colors with relevance of displayed information.^[1] The person searching might be given the color of the item to be found, or they may know the color of the sought category based on their experience. Alternatively, their task could require looking for an item that stands out as different, signified by color, with no target color given.

Salient colors might be used, for example, to highlight patterns or to enable rapid search:

• for a particular information-item or items on a geographic plot
  • among friendly, hostile and neutral forces on a military situation display^[2]
  • on a weather map^[3]
  • among geographically-displayed geologic, topographic, health, political, economic, demographic, technical or aeronautical data^[4]
  • for cartography^[5][6]
• in scientific visualization of
  • medical images
  • mathematical images
  • hyperspectral images
• along flow charts including
  • input flows, output flows, reservoirs, mixing chambers and condensation towers etc. in an industrial-process flow
  • project flow charts
  • process flow charts
  • sales flow charts
  • organization charts
  • financial flows, to follow the money
  • cause-and-effect diagrams
• within statistical visualizations like
  • scatter plots
  • pie diagrams
  • bar graphs
  • function plots
  • exploratory data analyses^[7]

Another application of color coding is to show symmetries in visualized data.^[8] Color coding can connect and untangle, e.g., trends on a plot, or the continuity of axons in the neural connectome. New applications of the innate skill to search, or group, based on color are still being invented, e.g., for networks,^[9] the Road Coloring Theorem, heat maps,^[10] the genome,^[11] genomic structural variation,^[12] genome browsers,^[13] and spatial data like molecules.^[14] Color coding has utility to visualize outliers and errors in unfamiliar data.

Color coding is used also for purposes other than visualization of displayed data. Data visualization complements, and does not substitute for or contradict, those other uses. Where conflict is possible, for example when there are established meanings of colors in other contexts such as signal lights, then those meanings should be given deference when choosing colors for a data-visualization color code.

1.2. Time Required to Find Visualized Information

Visual search is accomplished^[15] through rapid involuntary and subconscious eye movements called saccades. The eye makes about three saccades per second during visual search. Such eye movements constrain human cognitive performance.^[16] Search involves a neural network in the brain for processing motion and location, and controlling saccades.^[17] This neural network is sensitive to color and to change; color and flash-coding (temporally-varying brightness) can complement each other to enhance search speed, without mutual interference.^[18]

If there are N items to be searched in a display, N/2 saccades tend to be needed to locate a particular item.^[19] Given three saccades per second, N/(2*3) = N/6 seconds is an estimate of search time to find one item among N. The estimate is improved by adding reaction time, perhaps a second; expected search time equals 1 + N/6 seconds to find a target item among N displayed items.^[19] This estimate accords with empirical search times.^[20][21] In such a situation, color-coded search time increases linearly with the number of information items on a display that share the distinct target color.^[22] Furthermore, search is faster when the target-colored items are spatially organized, for instance in sinuous paths^[21] or presentation layout^[23] or design of data figures.^[24]

Search time has an exponential statistical distribution.^[25] In an exponential distribution, the variability (standard deviation) equals the expected value or average. As expected search time increases, so does the variability of search time. Longer expected search time implies a few very-long searches. For example, with 30 different search items (N=30) randomly distributed on the display, the expected search time is 6 seconds, but 5% of searches will last longer than 18 seconds. See Table 1 for similar results.

Short expected search time prevents unacceptably long searches associated with longer average search time. Based on the linear increase of search time with the number of potential targets, and the exponential distribution of search times (see Table 1), limiting displayed items sharing the target’s color to fewer than about 11 would keep search times to less than 10 seconds almost all of the time.^[4] More items could share the target color if the objective were to group subsets (different colors) of dispersed items for some purpose other than search.^[26]

1.3. Choosing Salient Colors for Color Coding

Color coding can make information salient^[27] (see Figure 1b). Discriminable color codes reduce or eliminate interference with search by all items not sharing the salient target color.^[28] Qualitative, subjective methods to choose distinct colors for coding are limited to a few colors.^[29] Color engineering achieved 22 paint colors of maximum contrast,^[30] before the advent of self-luminous color displays. Self-luminous devices, such as computer screens, are capable of higher luminances (wider color gamut), higher resolution (smaller symbols and image segments) and higher contrasts (more-salient distinctions from background) than are usually achievable with reflective materials like paint. These characteristics of self-luminous displays create color-coding opportunities and problems as discussed below.

An objective method of choosing distinctive colors for color-coded visualization of electronically-displayed data is to consider the color coordinates^[31] of the colors available within the display's gamut. Typically, a light-emitting display’s palette is described by three coordinates (R, G, B) each in the range 0 to 255. For instance, (255,0,0) should be a saturated red, (0,255,0) should be a green, (0,0,255) should be a deep blue, (255, 255, 255) should be a bright neutral and (0,0,0) should appear black. Choosing such extreme coordinate-triads for color coding is an approach to selecting distinct colors for color coding. Grays have equal R, G and B. Colors with nearly equal RGB coordinates are said to have low saturation.^[31] High color-saturation (i.e., no white light mixed with the spectral color from the hue circle) tends to enhance color salience. If more colors are needed, add (255, 255, 0) bright yellow, (0, 255, 255) blue-green, and (255, 0, 255) purple. The 15-color code in Figure 2 also includes reduced-intensity R or G or B: (127, 0, 0) dark red, (0, 127, 0) dark green, (0, 0, 127) dark blue, and two shades of gray, not including the intermediate gray background. Reduced intensity of the binary combinations also might be considered: (127, 127, 0), (127, 0, 127) or (0, 127, 127) the last of which is included in Figure 2 too. The range of (R, G, B) coordinates, 0 to 255, reflects an 8-bit display (2⁸ -1 = 255); other display technologies would have a different range of coordinates. The colors associated with RBG coordinates are particular to the display in use; they are not device-independent.

Figure 2. Some salient colors suitable for color coding are located on the outer boundary of the display's color gamut. The (R,G,B) coordinates of each color are shown in the corresponding colored hexagon. By RobertCCarter - MS PowerpointPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64542153

Figure 3. Schematic summary of the relation between apparent color similarity and calculated color difference. The red line is most relevant here. Other lines in Figure 3 will be discussed below. By RobertCCarter - Created using MS PowerpointPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64534371

Optical measurements can be used to calculate the color difference between each pair of colors in the color code. This involves spectro-radiometrically measuring^[31] the tristimulus values of colors available from the display. Tristimulus values (X, Y, Z) are device-independent color coordinates used to calculate color difference. Luminance, a correlate of grayscale, is the Y tristimulus value, for instance.^[32] The CIEDE2000, or CIECAM02 UCS, color difference formulas are statistically superior as metrics of large-color-difference discriminability.^[33] Generally, color difference calculations give more weight to red-green differences, and less to blue-yellow differences and luminance (i.e., gray scale) differences, in that order of efficacy.^[31]

Code colors having small color differences from the target color make distractor items look similar to the target color, as measured in terms of search time^[21] or in terms of eye movements.^[33] Displayed items with large color differences from the target do not interfere with target salience.^[21] Figure 3 summarizes this relationship between color similarity or salience during visualization and calculated color difference.^[33] The discipline of identifying (and separating, in color difference) the most similar colors in the color code will be useful, whatever method is used to choose code colors.

In a code with n colors, there are n(n-1)/2 pairs of colors (see Figure 4), each pair having a color difference.^[34] The proliferation of color differences, as the number of colors increases, necessitates a systematic, algorithmic and automated method of color selection.

Figure 4. The number of color pairs, and hence color differences, in a color code increases rapidly as the number of colors increases. By RobertCCarter - Created using MS ExcelPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64533270

Salience of many colors, seen in the context of each other, can be optimized. Because the smallest color differences interfere with search while large-enough color differences do not interfere (see Figure 3), the optimization objective^[35] is to maximize the minimum calculated color-difference, among the n(n-1)/2 color differences in an n-color code, in order to derive a large number (n) of distinct colors for a color code custom-fit to a particular display’s gamut. This basic method was elaborated for industrial use.^[36] The method has been applied to account for ambient illumination reflected from a display.^[37] The optimization method has been extended to alternative objective functions.^[38] It has been applied to complex display situations^[39][40] and to small symbols.^[41] Technical color measurement^[31] and optimized color difference can generate several alternative equally-effective codes that exploit the full gamut of the display and human color discrimination capability. This availability of alternative coding choices enables use of apropos colors with pre-established meanings or avoidance of inappropriate colors.

2. Practical Problems of Color Coding for Information Salience

2.1. Symbol Size Affects Color Salience

Apparent color difference between objects depends upon the visual angle of the objects that are viewed. Smaller symbols cause reduced color differences.^[41] Calculated color difference assumes a 2-degree visual subtense. The thumb-nail joint (from thumb-tip to joint-by-the-nail) appears about two degrees of visual subtense at arm’s length for example. Typical display symbols might subtend only 7 minutes, or less than 1/16 of two degrees.^[42] Consider this in the context of these facts^[43]

1. there are virtually no short-wavelength (blue) sensing cells on the retina within a 20-minute-diameter disk around the line of sight (LOS),
2. they comprise only about 7% of all daylight receptors and
3. the blue-sensing cells are about 4 minutes apart at their closest (within a degree of the LOS), increasing to about 8 minutes apart with greater eccentricity.

For these reasons, blue distinctions (involving colors on any tritan line,^[43]) are lost for small symbols. This is called small-field tritanopia, which is a color deficiency for blueness-differences among small visual fields, to which everyone is subject. Hence, blueness differences should be discounted (or even eliminated from consideration) when selecting coding colors for small symbols, less than about 30 minutes of (i.e., the sun’s and moon’s) angular subtense.^[44] Even color-differences not involving blueness appear to fade (however less than blue fades) when symbols diminish in size. For example, objects appear darker (less light) as they trend to a smaller visual subtense; larger areas with the same luminance appear lighter than smaller areas.^[41] It is possible to estimate the effect of visual subtense on apparent color difference, including grayscale difference.^[45] Smaller visual subtense of symbols implies fewer salient, or even discriminable, colors in the same display gamut.^[41] Code colors will be salient if their color differences are at least 14 CIEDE2000 units for symbols having a two-degree visual subtense, or the equivalent color difference as estimated for smaller subtense.^[45] For instance, salience of the 1.5-degree-by-0.75-degree visual-search targets used by Williams^[19] improved for increases of their color differences to at least 21 CIEDE2000 units; beyond about 21 their salience did not improve further.^[33] (The minimum color-difference values of salient colors given here correspond to the lower bound of the 95% confidence interval in the cited scientific literature.) The blue line and arrow in Figure 3 summarize the effect of small visual subtense (< 2 degrees of visual angle) on apparent color similarity, as a function of calculated color difference.^[42] The visual angle subtended by a displayed symbol or other image segment can be precisely measured or calculated.

2.2. Deficiency of Color Vision Can Affect Symbol Salience

The most common forms of inherited color deficiency involve red-green confusions.^[46] These color-perception confusions are due to a deficiency in retinal cells to sense long (e.g., red) or medium (e.g., green) wavelengths.^[43] Red or green would look darker for people deficient in the corresponding cells. Small visual subtense of symbols, or a rarer form of color deficiency (tritanopia), deplete blueness differences (due to sparse sampling by short-wavelength-sensing cells in the retina). A color code for general use, where neither color deficiency nor small visual subtense (e.g., distant viewing) can be avoided, is white, black, orange, and grays discriminable in luminance from the other code colors and from the background luminance.^[29] This assumes distant viewing by a person who retains the ability to sense either long or medium-wavelengths. Blue (but discriminably lighter than black) could be added, due to the rarity of deficient short-wavelength sensing cells, if viewing distances ensure visual subtense of color-coded symbols greater than 30 minutes. (See Figure 1b.)

Even grayscale perception can be affected by color deficiencies.^[47] Red-green-color-deficient observers who retain long-wavelength-sensing cells (deuteranopes) are accurate judges of color differences near red.^[47] Hence, the best color code for color-deficient observers would depend upon the type of color deficiency.

Images can be enhanced for viewing by color-deficient observers.^[48] The recommended methods are edge enhancement, recoloring and pattern superposition to complement color. Three kinds of images are envisioned: natural scene, scientific visualization or an office document. No technique satisfies the requirements of all applications.

Figure 5. A demonstration of how the luminance difference between a symbol and its background affects symbol legibility. The horizontal stripes and the vertical message-columns are each a constant color. By RobertCCarter - Created using MS PowerpointPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64526195

2.3. Legibility in the Context of Color Coding

Legibility is different from salience, discussed above. Legibility is related to clearly and distinctly seeing edges, shapes and spatial details, and therefore to reading. Legibility of a symbol depends upon luminance difference (not color difference) between the symbol and the background upon which the symbol is presented. Figure 5, the concept of which was presented by Australian architect Paul Green-Armytage,^[49] demonstrates this effect of luminance difference upon legibility. For this reason, yellow (a high-luminance color) is less legible on a bright white background, and conversely so are white symbols less legible on a yellow background. Dark blue (a low-luminance color) symbols are less legible on a black background, and conversely black letters are less legible on a dark blue background. This is despite yellow being salient on a white background and blue being conspicuous on a black background. Having adequate color difference among visualized items is complementary to, and compatible with, enabling legibility with adequate luminance difference between the items and their background. There are resources for quantifying the required luminance difference to enable legibility.

3. Grayscale, an Important Tool for Visualization of Data

In addition to its effect of on legibility, luminance also affects grayscale. Lightness and brightness differences are grayscale attributes of color difference, so they can affect conspicuousness of symbols. Grayscale is accessible to people with color-vision deficiencies,^[47] and it is less susceptible (than chromatic differences) to fading due to small visual subtense.^[45] Grayscale (rather than chromatic color difference) has been shown to be particularly apt for coding ordinal data such as temperature on a weather map.^[3][50]

Changing the background luminance makes salient the difference between the (possibly color-coded) symbols having luminances just greater than, and just less than, each background luminance selected. The dotted gray line in Figure 3 summarizes this effect. Leon Williams,^[51] observed this and suggested a data-visualization technique he called data slicing, as exemplified in Figure 6.

Figures 6a-d. Data slicing, or changing the background luminance of an image, can render different data more salient. There are four copies of the same data, each copy with a different background luminance. Click the figure to see full-screen details. By RobertCCarter - Created using MS PowerpointPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64533027

The Whittle logarithmic grayscale calculation^[52] for self-luminous devices quantifies visual effects of luminance differences among displayed symbols. It enables calculation of any number of equal perceptible differences^[53] (nEPD) as small as the threshold of visibility, or of any suprathreshold magnitude. The unit size of nEPD is three or four times the absolute threshold of visibility for luminance change. This nEPD unit is “just noticeable at a glance” with free viewing of an electronic display.^[52]

The Whittle calculation is unusual (among grayscale formulae) in four ways.

1. It includes the background luminance.^[52]
2. It applies to all photopic (i.e., daylight) luminances, based on data for thresholds^[53] and matching.^[54]
3. Its derivatives are meaningful.^[55]
4. It can account for the spatial scale of the contrast.^[52]

The derivative (slope) of nEPD with respect to background luminance implies an optimum background luminance.^[56] White backgrounds are used for historical reasons related to legibility of small symbols. However, there is no symbol size or symbol-luminance range for which a white background is optimum, in the sense of maximizing the number of visibly-different gray shades that symbols can have in this luminance range. The optimum background can double the number of gray shades visible among larger symbols. The optimum background luminance is always less than 46% of the maximum of the symbol luminance range, for ranges including zero.^[57]

Contrasts (e.g., displayed symbols) have a luminance difference from their background.^[52] The contrast luminance would be less than the background luminance for negative contrasts and the contrast luminance exceeds the background luminance for positive contrasts. The derivative of nEPD with respect to contrast luminance predicts the threshold of contrast visibility (e.g., to prevent banding in a digital image) for any daylight combination of symbol and background luminance.^[53][58] The Whittle calculation’s derivative with respect to contrast luminance also quantifies human suprathreshold sensitivity to luminance differences, or changes, for all combinations of daylight target and background luminance.^[55] This derivative is germane to digital image processing, and particularly histogram specification,^[59] suggesting applicability to medical-image windowing. The Whittle calculation shows the magnitude of positive contrast beyond which contrast brightness increases (with increasing contrast luminance) as fast against any background as against an unilluminated (black) background.^[55] At lesser positive contrasts, and for negative contrasts, symbol lightness depends upon the background luminance of the symbol (in ways described by the Whittle calculation, and known as lightness constancy). Display luminance-contrast calibration^[60] would be necessary in mission-critical applications.

Figure 7. Plot of the Whittle grayscale calculation (adopted by CIE^[61]) for self-luminous devices, as applied to an optotype E subtending 10 arc minutes (1/3 the diameter of the sun or full moon). See text for details about this plot. By RobertCCarter - Created using MS PowerpointPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64532552

In Figure 7, there is a different nEPD curve for each background luminance. The steepest part of each curve (most change of grayscale per unit change of symbol luminance) is for symbol luminances almost equal to their background luminance. Another relatively steep part of each curve is at the extreme left, where symbol luminance is near zero.^[55] Negative nEPD corresponds to negative contrasts and positive nEPD corresponds to positive contrasts. Symbols with equal nEPD from their respective backgrounds will appear the same shade of gray,^[62] despite being on different backgrounds. (See Figure 8, which illustrates this matching of grays seen on different backgrounds.) The logarithmic curvature of the Whittle calculation has a rational basis.^[63] An intermediate background luminance for Figure 7 itself increases the number of grayscale steps visible among the curves, making plausible an optimum background luminance.

The Whittle calculation for self-luminous devices has a parameter, k, relating to the spatial scale of the symbol.^[55] It (k) is the proportion (between 0 and 100%) of image contrast (target luminance minus background luminance) lost to intraocular scattering on the way to becoming a retinal image.^[52] Reducing the angular subtense of a symbol always reduces the physical luminance difference between the symbol and its background (due to intraocular scattering), hence reducing legibility and conspicuousness and increasing k.^[52] This parameter, k, can be calculated from optical principles,^[57] as it was for the gray scale of the 10-arc-minute (k=0.2) optotype E represented in Figure 7. When symbol angular subtense (represented by k) is reduced, the optimum background luminance also is reduced.^[57] The Whittle calculation is unaffected by highlights remote from the contrast, as verified experimentally.^[64] The effect of remote highlights on grayscale appearance is due to intraocular scattering.^[55]

Figure 8. This figure illustrates the concept of matching grays with different background luminances. Each hexagon has a different luminance. More-exact matches could be calculated for a specific viewing situation. By RobertCCarter - Created using MS PowerpointPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64647780

Figure 9. Identical wreaths of six grayscale hexagons having different background luminances. A more-luminous background darkens the grayscale. By RobertCCarter - Created using MS PowerpointPreviously published: No prior publication, CC BY-SA 4.0, https://en.wikipedia.org/w/index.php?curid=64532396

A grayscale phenomenon calculable with the Whittle calculation for self-luminous devices is matching grays having different background luminances.^[54] Conversely, the calculation can be used to find the background luminances that will make specified contrast luminances match. According to the calculation (as can be seen in Figure 7), it is impossible to match negative contrasts with positive contrasts; negative contrasts always look darker than positive contrasts. Three matching (positive-contrast) clouds in Figure 8 have greater luminance than their respective background hexagons. Three (negative-contrast clouds) have less luminance than their background, and they match each other too.

According to the Whittle calculation, darker grays are achievable with a more-luminous background.^[52] The effect is more pronounced for larger subtenses, but can be seen in the lower left of Figure 7 where more-negative nEPD is achieved when the background luminance is greater. Figure 9 exemplifies the effect; the same hexagonal visual target is seen as a darker gray (including black) when its background is more luminous, according to the Whittle calculation.

4. Ongoing Research and Development

Research and development continues on data visualization technologies. For instance, there is ongoing research and development to improve color difference calculations.^[65] The primary motivation for this R&D is determination of minimum detectable thresholds of color difference, for quality control over industrial color processes (e.g., food, paint, textiles, plastics, print and self-luminous displays). As color difference calculations have been improved for threshold determination, they fortuitously have become better for the suprathreshold applications related to data visualization.^[33] Ever since the relationship in Figure 3 was first shown,^[28] there has been practical interest in more-precise determination of the (calculated color-difference) location of the knee of the curve for two-degree visual subtense and for smaller subtenses.^[45] A general societal concern for inclusiveness, and particularly for accommodating disabilities, has stimulated research on color applications for people with color deficient vision.^[48] A vibrant area of research (for instance by Brian Wandell and David Brainard,^[66] and CIE^[65]) is computational modelling to predict (e.g., color) appearance, including all factors such as intraocular optics, retinal cone-cell responses, effects of visual subtense, eccentricity from the line of sight^[67], atmospheric optics, the illuminating light spectrum, context and color adaptation. There is research of methods to calibrate displays,^[68] for example to make colors salient on a display located remote from the person programming the data visualization, and to more-accurately render color on displays.^[69] High dynamic range (HDR) displays enable the most-salient colors for visualization; HDR is an active area of research.^[70] Research continues on visual pathways and circuits in the brain;^[17] some of this is relevant to data visualization and comprehension. Proprietary developments of color-display technology, computers and software improve data visualization, too. There is research to apply artistic color-coding techniques to visualization "to enable nonscientists to work with actual data to communicate issues that are critical to humanity."^[71]