Traffic sign pattern recognition algorithm

The TSPR project aims to develop the pattern recognition tools specifically for the traffic signs used by the U.S. Department of Transportation Federal Highway Administration. The signs in our interests are defined in the Manual on Uniform Traffic Control Devices (MUTCD). MUTCD is the national standard for all traffic control devices installed on any street, highway, or bycycle trail open to public travle as proivded in Title 23 of the Code of Federal Regulations, Part 655.603. Our group provides the extensive analysis and manually constructed traffic sign database of MUTCD at The summary of MUTCD.

This research is under progress by the Center for Geographic Information system in Georgia Institute of Technology. For details on research members, please visit the TSPR Project Team page. Detecting signs in natural roadway is one of the toughest challenges in computer vision. Detail problems are enlisted at Challenges in the TSPR project. Reviews on related works and progresses are available at Reviews on TSPR related projects.

Our approach is featured using Neural Network which is specially categorized and trained for the each class of MUTCD signs. Details are at Our Approaches. Detail schedules are shown at the Schedule Table and check To-do List for the job details. Progress reports will be archived at Progress Reports and please use Discussions for the Q&A and suggestions. All deliveries of this project are available at Download. If you are interested every single media including files and images, then please access Special:Imagelist.

MediaWiki for the traffic sign pattern recognition (TSPR) project has been successfully installed. Consult the User’s Guide for information on using the wiki software.

Recent News

mutcdcodesearch.png]]

  • 2006 International Conference on Image Processing
  • 2006 International Conference on Image Processing held at Atlanta, Georgia from last Sunday (10/08/2006) to today (10/11/2006). Thanks to Dr. Tsai, I could have participated in this conference as a guest and it was very enjoyfull experience. ICIP is a pretty big conference. This year it holds 834 technical papers and posters. I took many lessons and talked with a number of authors in poster sessions in person. My short reviews, which will follow hereinafter, will cover specific research areas of our interests.

  • Adaptive HO-HSV Traffic Sign Color Model
  • Abstracts A new color model designed exclusively for the traffic sign color recognition is presented in this paper. It featured using the high-order of HSV entities to reflect the relationship between various viewing conditions on the road and the traffic sign color appearance. Our model named an adaptive high-order HSV model is the formulation of the ground-truth traffic sign color probabilities built using the extensive United States department of transportation database.

  • Challenges in the TSPR project
  • Excerpts from Fang et al. Road signs are typically placed either by the roadside or above roads. They provide important information for guiding, warning, or regulating the behaviors drivers in order to make driving safer and easier. The difficulty in recognizing road signs is largely due to the following reasons: The colors of road signs, particularly red, may fade after long exposure to the sun. Moreover, paint may even flake or peel off.

  • Choose proper image abstraction or feature for sign recognition
  • In order to develop a system that recognize and classify the different types of input patterns, a number of features or abstractions of a source image should be computed first and tested for the classification. However in practice, it is difficult to find out how many feature extractors should be applied to categorize many different types of images automatically. In our approach, it can be easily understood that one single feature of an input image may not be applicable to categorize 328 different types of traffic sign patterns.

  • Choosing a proper color space
  • In traffic signs, colors whether used in the background or legends are very important in classifying the traffic sign categories. This page will provide the basic introduction on color space and further investigate to choose the proper color space for the traffic sign color decomposition. Color spaces Image:rgb_space.png|RGB Image:hsv_space.png|HSV Image:yuv_space.png|YUV Image:ycbcr_space.png|YCbCr Image:cmy_space.png|CMY Image:lms_space.png|LMS Image:xyz_space.png|XYZ Image:lub_space.png|L*u*v* Image:lab_space.png|L*a*b* MUTCD color specification The USA FHWA (Federal Highway Administration) has color specifications for retroreflective signing materials which were first developed in the late 1960’s and in 2002, with the technological advances in the manufacturing of signing and markings materials and the measurement of color, they were revised as shown in Table 2 and Table 3 which are daytime color specification limits for retroreflective material.

  • Construct the traffic sign database
  • Current tagged traffic sign database Regarding the database schema and their relationship, please see below sections Statistics (09/15/2006) Total tagged traffic signs: 3446 Distinct codes: 96 Sign categories: 8 MUTCD code database The official release of USA traffic signs can be found at MUTCD. They offer sign resources in HTMLs and PDFs so it requires efforts to extract the sign images from their documents. We manually extracted 673 MUTCD code signs and provides those in the form of database with search interfaces.

  • Develop traffic sign recognition modules
  • ANN (Artifical Neural Network) technologies have been developed for a long time since its birth and so there are a number of open source packages for the research purposes. Within those two packages are compared considering its learning speed and computing times. Our first experiment will be focused on the OCR case and later we will extend the selected ANN module for the traffic sign recognition. Traffic sign feature extraction Please see Traffic sign feature extraction

  • Download
  • MUTCD Code Search A simple program interface to search the MUTCD code database. Please see MUTCD Code Search Interface. MUTCD Traffic Sign Coder Traffic sign coder helps users to find the location of the traffic sign in a given image and assign the MUTCD traffic sign codes. Please see MUTCD Traffic Sign Coder for more details. Download Media:mutcd_trafficsigncoder_09072006.zip Note: MUTCD database for the program is not released officially. So if you are not in the developer team, this program will not run.

  • Estimating the minimum geographical distance for the traffic sign recognition
  • Question: Current images are taken at every 10.98 ft. Based on the results, can we suggest that the maximun spacing of taking images (e.g. 20 ft)? I guess your question in other words whether our system can recognize the traffic sign as far as 20 fts away. However the image size in width and height that we can extract from the image depends on the various camera parameters. We do not have such information now to do the camera calibration to compute the expected pixel size of the traffic sign.

  • Extending line receptors
  • Currently, line receptors are also initially generated randomly. Although we use the maximum-entropy theory to filter out the most informative set of line receptors, the FCG graph can be applied later as the measure of the line receptor set correlation.

  • Extension for the traffic sign recognition
  • As we showed in Choose proper image abstractions or features for sign recognition, simple image features are not proper to classify every kinds of traffic signs. Analogously, although the concept of the line receptor using those as an image abstraction feature provides the robust generalization of the binary image pattern but its performance should be carefully checked when the target image patterns, in our case traffic signs, are complex and when the multiple colors are also used to represent its pattern.

  • Finding a roadway lane using video processing
  • Pilho Kim, ECE GaTech 12/02/2005 1. Objective Find roadway lanes from a real driving video. 2. Preparation Step 1. Install a camcorder. (Be very careful to make it stable.) Step 2. Capture! 2005-12-02 9:05 am (Click here to download the video) 2005-12-01 8:25 pm (Click here to download the video) 3. Analysis Step 1. Edge detection test. (12/02/2005) A test image extracted from James Tsai’s paper Canny edge detection Step 3.

  • Fonts used in FHWN
  • The official name for the font most commonly used on US road signs and highway maker shields is FHWA (FHWA referes to the Federal Highway Administration.) Series E modified. This is a sana-serif font. For details, please visit Traffic sign from Wikipedia. A commercial font is sold by DGI-Technologies. A free font, not for the commercial purposes, is also available at Roadgeek Fonts.

  • GPS reader
  • GPS reader is the software to read GPS data from the GPS sensor through the serial connection. A GPS sensor should provide the serial communication whether it is the real RS-232 connection or through USB-to-Serial port mapping driver (Thanks to Dr. Wang for this finding). If a user have a GPS sensor with the USB support, then he should find a driver file for this purpose. The raw GPS data that you can get through the serial connection to a GPS sensor is shown below.

  • Members
  • Members Main developer: Pilho Kim Postdoctoral fellow: Dr. Zhaohua Wang Advisor: Dr. Yichang (James) Tsai

  • MUTCD code search interface
  • Introduction This program provides the database interface to search the Manual on Uniform Traffic Control Devices codes. Each sign image is excerpted from the PDF manual. Users can specify the search option at the Query options box. The properties of each sign will be displayed when a user select a sign. Additionally, to see all signs in the database (currently 667 types), press Show all signs at the bottom of the query window.

  • MUTCD Database
  • Sign images are collected from the Manual of Traffic Signs by Richard C. Moeur. All traffic sign images captured and used under should follow the owner’s Standard Use Agreement. After downloading, extract a zipped file to your directory and run Start.lnk file. Download: Media:TrafficSignImageDatabase.zip After downloading, extract a zipped file at your directory If you want various sizes (200x200, 100x100, 50x50, 25x25) in two image formats(gif, bmp), then download this. After downloading, extract a zipped file to your directory.

  • MUTCD summary
  • MUTCD manual For details on MUTCD, please visit Manual on Uniform Traffic Control Devices. The complete PDF version of MUTCD manuals in one file is compiled for convenience (In MUTCD web site, PDF files are separated chapter by chapter). Download MUTCD2004revision.pdf References MUTCD color specifications Fonts used in FHWN

  • MUTCD traffic sign coder
  • Revision history 09/06/2006 Initialize button to set keywords and options in the Search MUTCD codebook window to its initial status. Now a user can move bewteen options in the Search MUTCD codebook using the TAB key to move down and Shift-TAB to move back. It is possible now to embed the database within the program. This means that we do not need to install MySQL database separately. I embedded the MySQL source codes into the MUTCD Traffic Sign Coder program.

  • Neural network
  • A neural network is an interconnected group of biological neurons. In modern usage the term can also refer to artificial neural networks, which are constituted of artificial neurons. Thus the term ‘Neural Network’ specifies two distinct concepts: A biological neural network is a plexus of connected or functionally related neurons in the peripheral nervous system or the central nervous system. In the field of neuroscience, it most often refers to a group of neurons from a nervous system that are suited for laboratory analysis.

  • OpenCV libraries
  • Before we start developing any computer vision programs, it deserves efforts and times to search and select proper image processing libraries. One of the prorgammers developing IM, a handy and very well encapsulated and resusable image processing library, compared theirs with other famous libraries including OpenCV and VTK at 1. I reviewed it with my personal opinions in the folowing. Reviews At the end of the above page located under “Comparison” paragraph they mentioned several points.

  • Our approaches
  • For details, please click each item. Construct the traffic sign database. Choose proper image abstractions or features for sign recognition. Develop traffic sign recognition modules. Result analysis for the further improvement. Conclusion.

  • Publications
  • James Tsai, Pilho Kim and Zhaohua Wang, Generalized traffic sign detection model for developing a sign inventory, Journal of Computing in Civil Engineering, vol. 23, no. 5, pp. 266-276, 2009. ASCE J. Wu and Y. J. Tsai, Enhanced Roadway Inventory Using a 2-D Sign Video Image Recognition Algorithm. Computer–Aided Civil and Infrastructure Engineering, vol. 21, issue. 5, pp. 369-382. CACIE Y. Tsai and J. Wu, Shape- and Texture-Based 1-D Image Processing Algorithm for Real-Time Stop Sign Road Inventory Data Collection, ITS Journal (Intelligent Transportation Systems), vol.

  • Result analysis for the further improvement
  • Our current preliminary TSR (Traffic Sing Recognition) implemenatation is based on the bianarized image which uses the dominating color as its background, thus black, and all other colors as the forground, white. After we apply the line receptors above on it to compute the crossing state to use as the input for the trainned ANN. However, the binarized image in fact does not contain the important color information and hence the preliminary test results also reflects this point.

  • Reviews on TSPR related projects
  • To see all references under review, please visit our CiteULike page. Related researches Traffic sign recognition has been an important topic in various research areas including, but not limited to, video surveillance, GIS information collection, and federal asset management. While numerous computer-vision based approaches are developed toward traffic sign detection (See Road signs recognition - survey of the state of the art), the methods for traffic sign recognition, which is identifying the exact class of traffic sign, one of the popular approaches, has been heavily depending on the artificial neural network considering.

  • Schedules
  • Schedules 04/28/2006 Project launching meeting (Dr. Tasi, Dr. Wang, Pilho – namly All members): Discuss the topic of the project (MUTCD) for Pilho’s summer RA. 05/04/2006 Regular meeting (all): The objective of this meeting is to discuss with the preliminary thought that Pilho have on developing the NN and Dr. Wang will provide Pilho with the input ROIs.Continue reviews on existing approaches. Manually select tens of warning sign candidates from MUTCD databases.

  • Select a number of sign candidates for a priori algorithm testing
  • Within extensive traffic signs, we first focus on the warning signs. From 118 different types of warning signs, 20 candidates are manualy selected. .

  • Selecting a proper ANN module
  • Selecting a proper ANN module Preparing training data The learning data set for the neural network training has special formats. The data set here shows an example for the FANN training and its first row of the data includes the number of the training data sets (26), the number of inputs (100), and the number of outputs (26). The next row has 100 input data and the following row includes the output teaching data.

  • To-do list
  • To-do list How to detext the traffic sign from the natural scene? Is it possible to seperate the traffic sign detection algorithms and traffic sign recognition algorithm? Extract the foreground and background from the given traffic sign image. Detect the boundary of a traffic sign and seperate the background and foreground contents.

  • Traffic sign color information processing
  • Introduction Traffic signs are designed to attract driver’s attention by its color, shape, and contents painted inside of the sign. Within above traffic sign features, color plays a dominant role in discriminating the type of signs. For instance, the background color of the traffic sign represents its class or type of the sign like guide, recreational, warning, or regulatory signs. The USA FHWA (Federal Highway Administration) has color specifications for retroreflective signing materials which were first developed in the late 1960’s and in 2002, with the technological advances in the manufacturing of signing and markings materials and the measurement of color, they were revised as shown in Choosing a proper color space#MUTCD color specification Table 2 and Table 3 which are daytime color specification limits for retroreflective material.

  • Traffic sign feature extraction
  • Traffic sign feature extraction It is very important to choose proper image features for the pattern recognition. However, this feature extrating algorithm should only be applied to our region of interests (ROIs). For this, we composed the pre-processing job for traffic sign recognition into several sequential steps as shown in the following picture. 1st classification: Traffic sign boundary detection We first develop a tool to detect and extract the traffic signs from a given natural scene capture.

  • Traffic sign pattern ANN training algorithm
  • This shows the preliminary TSPR training algorithms.

  • Traffic sign pattern recognition algorithm
  • Traffic sign pattern recognition algorithm Actual traffic sign pattern recognition is composed of several steps as shown in the following graph. This shows the preliminary TSPR recognition algorithm.

  • Traffic sign recognition test
  • For the traffic sign recognition test, the traffic sign recognition user interface named Traffic Sign Recognizer (TSR) is developed. The new GUI for traffic sign recognition is being built on top of the polygon detection algorithm that I introduced at Traffic sign feature extraction. The manual region selection function is additionally introduced. Below screen shot shows one example that TSR recognized the school warning sign successfully. Image:tspr_extracted_region_after_boundary_removal.png|Extracted region after boundary removal Image:tspr_resized_image.

  • Weekly reports
  • 2006-5-12 During this week, I collected traffic signs on the web and finally found a very nice complete database maintained by Richard C. Moeur, who is a voting member of the National Committee on Uniform Traffic Control Devices (NCUTCD). I zipped his database and uploaded at my site for your information. You can download the database at Construct the traffic sign database. Please be advised that Moeur had placed a copyright rule limited to recreational or study purposes.