Extension for the traffic sign recognition
As we showed in Choose proper image abstractions or features for sign recognition, simple image features are not proper to classify every kinds of traffic signs. Analogously, although the concept of the line receptor using those as an image abstraction feature provides the robust generalization of the binary image pattern but its performance should be carefully checked when the target image patterns, in our case traffic signs, are complex and when the multiple colors are also used to represent its pattern.
Preliminary traffic sign recognition using line receptors
A below image shows the simple binarization using its dominant color as
its background, 0, and other colors as informative, 1.
. This binarization
has some problem when a same color used in the background is used again
to draw the information inside the traffic sign. For an example, W3-3
warning sign (W3-3.jpg in the image) after binarization
(W3-3_binarized.jpg) does not have the signal lamp shape.
Small set training and analysis
Within above test set, 10 sample traffic signs shown below are selected
for immediate algorithm test.
.
- Line receptor generating algorithm:
.
- Summary
- Randomly generated a number of line receptors.
- Computes the state of each receptor (1 means the receptor crossing the traffic sign and -1 for not). So each line receptor will have an array of the number of the input pattern size (in this example, 10) filled with a value, 1 or -1, depending on its crossing on each input pattern.
- Remove duplicated line receptors based on the line state. If one line receptor has the state array after cross checking, for instance {-1 -1 1 1 1 -1 -1 -1 -1 -1}, and other line receptor has the same state values, then one receptor will be removed.
- Apply entropy maximization theorem to filter only 50 line receptors from receptors after duplication removal procedure.
The results after applying alogrithms to the small test set are shown in the following pictures.
Image:tspr_initial_receptors.png|Randomly generated 500 line receptors Image:tspr_duplicates_removed_receptors.png|81 left after removing 419 duplicated line receptors. Image:tspr_final_receptors.png|Selected 50 line receptors after maximizing entropy.
Final 50 line receptors are used as the input to the ANN. At this time,
ANN is configured to have 50 inputs (the crossing state of each line
receptor), 10 outputs indicating the ID of a recognized input pattern,
one hidden layer, and 300 hidden nodes. The ANN training was done
successfully as shown in the following ANN learning graph. The ANN
training took 0.28 second at the laptop with Intel Pentium P4 2.66GHz
CPU having 512MB memory.
In this experiment, it should be examined carefully why so many 419 line receptors are removed. This basically depends on the complexity and variance of its input training patterns. As shown in the above picture, only 10 distinguished traffic signs are binarized as the input pattern at this time. And some of they have a similar boundary shape with others. This makes line receptors have again similar crossing states and this have them filtered out. If input patterns are more distinguished and complex, then there will be less duplicated ones even using the same number of line receptors. This assumption is proved to be true in the following section.
Source codes updates
Please overwrites to the existing TSPR libraries. Media:tspr_library_update_06242006.zip
ANN training graph file Media:fann_learning_curve.xls
Full traffic sign set ANN training
Let us now extend the target pattern to the whole set of traffic signs (385 types).
Image:tspr_fullset_initial_receptors.png|Randomly generated 500 line receptors Image:tspr_fullset_duplicates_removed_receptors.png|499 left. Only 1 duplicated line receptor found. Image:tspr_fullset_final_receptors.png|Selected 300 line receptors after maximizing entropy.
In this experiement, the complexity of the input patterns, 385 traffic signs, is shown very well with the number of removed receptors. Only one receptor is removed in this case. Within those considering the size of ANN network, 300 line receptors are selected and the new ANN is trained. ANN is configured to have 300 inputs (the crossing state of each line receptor), 385 outputs indicating the ID of a recognized input pattern, one hidden layer, and 600 hidden nodes. The ANN training was done successfully as shown in the following ANN learning graph. The ANN training took 571 seconds at the laptop with Intel Pentium P4 2.66GHz CPU having 512MB memory.
Adpative training to determine the proper number of layers and nodes
Cascade training differs from ordinary training in the sense that it starts with an empty neural network and then adds neurons one by one, while it trains the neural network. The main benefit of this approach, is that you do not have to guess the number of hidden layers and neurons prior to training, but cascade training have also proved better at solving some problems.
The basic idea of cascade training is that a number of candidate neurons are trained separate from the real network, then the most promissing of these candidate neurons is inserted into the neural network. Then the output connections are trained and new candidate neurons is prepared. The candidate neurons are created as shorcut connected neurons in a new hidden layer, which means that the final neural network will consist of a number of hidden layers with one shorcut connected neuron in each.
Analysis
More than we expected it would be, our ANN is trained very well showing zero error for the complicated 325 input patterns. However, input patterns used in this experiment are computer generated synthetic images having very clear colors and boundaries. So it is time to move to real traffic signs captured on the road for the ANN testing and if necessary, more training.
Download ANN training data and trained ANN data
Extending traffic sign categorization based on dominating features
From traffic signs as shown in the above picture, a proper set of dominating features should be carefully selected and computed in the level of high accuracy for the successful categorization. Many image abstraction techniques may be applied to the traffic sign but within those, we select following three features as the preliminary traffic sign features.
- Boundary shape: triangle, rectangle, pentagon, hexagon and circle
- Foreground color: Based on MUTCD specifications, only seven colors (white, black, red, brown, green, dark blue and yellow) are used to draw traffic signs
- Background color: Within seven colors, white, green, brown, yellow and dark blue colors are used in the sign background.
Depending on the complexity of the training patterns, above features may not be enough to create the clear categories for ANN to recognize. We may introduce additional features like the presence of texts, specific shapes located within the signs, and OCR results.
Correlation comparison results based on each categorization class
We will show the performance of each categorization using the feature correlation graph (FCG).