Traffic sign color information processing

Introduction

Traffic signs are designed to attract driver’s attention by its color, shape, and contents painted inside of the sign. Within above traffic sign features, color plays a dominant role in discriminating the type of signs. For instance, the background color of the traffic sign represents its class or type of the sign like guide, recreational, warning, or regulatory signs.

The USA FHWA (Federal Highway Administration) has color specifications for retroreflective signing materials which were first developed in the late 1960’s and in 2002, with the technological advances in the manufacturing of signing and markings materials and the measurement of color, they were revised as shown in Choosing a proper color space#MUTCD color specification Table 2 and Table 3 which are daytime color specification limits for retroreflective material. Table 2 has 4 coordinates in x and y components of CIE Yxy color space. These 4 coordinates are vertices of a closed polygon located in the Yxy color space. Therefore any traffic sign printing company should print each color within the specified color range. The luminance factor is stored separately in Table 3 to handle different type of printing materials.

The problem in the context of road traffic sign recognition is that the illumination assumption in their specification is D65, representing normal daytime lights. Basically, FHWA suggests the guide line for the printing company in a limited condition and they are not suggesting the standard how a printed color should be appeared to drivers. In fact, these colorimetric transformations do not determine how the appearance of a color stimulus, specified in terms of its colorimetry, would change if it were moved from one set of viewing conditions to another, in our case the road condition. Viewing conditions may include following factors:

  1. Viewing flare contribution to observed stimuli
  2. Stimuli absolute luminances
  3. Observer chromatic adaptation
  4. Observer lateral-brightness adaptation
  5. Observer general-brightness adaptation

The limitation using the FHWA color specification for classifying the traffic sign color of the load sign image can be easily identified from the conversion result shown in the below figure. The objective of this experiment is finding the most closet one of 10 MUTCD colors . Cyan, which is not the MUTCD color, in the images means the background or the non-MUTCD color pixel. So the output image is expected to have only 11 colors: 10 MUTCD colors and Cyan for non-informative or background pixels.

color_conversion_comparision_1.png

The second row in the image is the FHWA color specification based color conversion result. As shown in the result, almost all pixel colors in a image captured on the road are out of color spectrums in the FHWA color table. This is because on the road viewing conditions are very different with D65 in the CIE standard. The third row is the color recognition results using our Color-PDF based conversion result. Our method will be explained herein after.

Based on the MUTCD manual on colors, 10 distinct colors are currently used to print the traffic sign. Although color plays a dominant role because of its discriminative power, predicting the correct ranges of hues and luminescence is an ill-posed problem due to lacks of viewing conditions. Basically the color appearance heavily depends on the various environment factors.

However, this environment that affects the color appearance can not be modeled mathematically in the traffic sign cases because the information on the luminosity, the number of light sources, its incidence angle, interfering objects and their colors that affect the human color recognition are not available in the collected MUTCD images. Even assuming that those are available, any practical color model to accommodate viewing condition factors has not been established yet. Therefore, it makes sense to use a learning approach to model the color description. In this way, the application dictates the allowed degree of variation.

Our approach

In this paper, we propose a new color segmentation method using color PDF (probability density function) to differentiate MUTCD colors in a specific sign using actual sign images taken on the roadway. The color PDF defined is built manually by marking sign regions color-by-color from 1893 DOT traffic images in which traffic signs exist. Color-marked sample images are displayed at the below image. This is approximately 5% of total 37,686 video log images.

colormask_to_collect_color_profile.png

As a result, selected 8 color PDFs are constructed. The below table includes the number of distinct RGB values of each pixel classified by human to have the same color. The folowing figure shows a portion of actual RGB values in the collected samples.

distinct_color_values.png

color_profile_upper_probability.png

The problem in applying the color PDFs for the MUTCD color recognition is clearly visualized in the below figure in which each color PDF is significantly interfering with other color areas even in case of selected 8 MUTCD colors. In practice, this occurs very commonly at the boundary of different signs because the image color quantization algorithm embedded in the image capturing system generally consider the surrounding pixel colors to choose the best candidates color value.

hsv_profile_fullset_bounded.png

This interfering problem is why we introduced the concept of color probability of a specific RGB value to be one of the MUTCD color. In our approach, a MUTCD color of each pixel is decided based on the occurrence of each specific RGB color value as the criteria. It can be easily extended to consider neighbor pixels, their PDFs and spatial relationships. This topic is left for the next phase development.

Implementation

Color PDF construction

color_pdf_fnn.png

Trained result samples are shown below. color_pdf_weight_black.png

Sample MUTCD color detection

  • STOP and NO PARKING signs

R8-3a_922_image.jpg R8-3a_922_image.png

  • School sign sample

S1-1_1547_image.jpg S1-1_1547_image.png

  • Multiple signs sample

OM-3R_2289_image.jpg OM-3R_2289_image.png

Statistics

  • Distinct colors
  1. Black
  2. Blue
  3. Fluorescent Yellow-Green
  4. Green
  5. Orange
  6. Red
  7. White
  8. Yellow
  • Duplicated distinct RGB values assigned to more than one MUTCD color.
    • The sum of distinct RGB counts of each MUTCD colors (8): 548897
    • Total distinct RGB counts: 413724
    • Overlapped RGB counts: 135173 (24.6%)