1. Home
2. Ideas
3. Machine Vision Interface with Light Emit...

Machine Vision Interface with Light Emitting Diode (LED) Traffic Control Device Displays

• Aug 6, 2020

Research Description

This idea was submitted on behalf of AASHTO Special Committee on Research and Innovation:

JoLight Emitting Diode (LED) Traffic Control Device (TCD) displays utilize pulse width modulated drivers to stabilize the color matrix and control both light intensity and heat build-up. The individual LEDs or blocks of LEDs in a display rapidly flash on and off at a rate invisible to the human eye. In some full color matrix displays that contain Red, Green and Blue (RGB) LEDs the individual colors flash on and off at different rates. Preliminary information indicates a 60 – 240 Hertz (Hz) cycle in existing LED TCDs including pavement markings, signs and signals. In addition to TCDs, LEDs are prevalent in vehicle lighting (headlights, turn signals, brake lights, etc.) systems.

In an Automated Driving Systems (ADS) cameras are a sensor component capable of detecting an electronic display TCD and recognizing the message provided. ADS camera shutter speeds are typically set at high rates to avoid motion blur. The variation between the shutter speed of the detection and the refresh rates of the LED TCDs can lead to blank panels or display elements.

LED TCD displays are a critical element of the nation’s transportation system management operations (TSMO) infrastructure. LED TCD displays have a five (5) year for portable equipment to twenty-five (25) year life cycle for larger investments (such as overhead signals and dynamic message signs). Various reports on the nations automobile fleet replacement cycle indicates that the typical vehicle on the roadway is approximately 12 years old . Considering the long lead time for conversion of the nation’s automotive fleet and its transportation infrastructure, NOW is the ideal time to develop a collaborative, industry-supported ADS Machine Vision Interface with LED TCD displays. The solution could be embedded into several relevant standards and regulations that are currently scheduled to be updated including:

• National Electronic Manufacturers Association (NEMA), TS 4 Dynamic Message Sign standard • American Association of State Highway Transportation Officials (AASHTO) A Policy on Geometric Design of Highways and Streets (Greenbook), and • United Stated Department of Transportation (USDOT) Federal Highway Administration (FHWA) Manual on Uniform Traffic Control Devices (MUTCD)

This research effort is needed immediately to support the scheduled standard updates. Waiting ten years for the next update cycle to begin would negatively impact the safe deployment of TCDs compatible with automated vehicles within the normal equipment replacement cycle.

Additional Supporting Information

Literature Search Summary TRID Search terms: “Machine Vision” and “LED” , “LED Flicker” and “ADS Machine Vision” produced no recent and directly relevant projects. The existing body of research does not address the relationship between the image creation (LED) and the image capture and processing needed for automated vehicles to respond appropriately to LED traffic control device displays.

4. Research Objective The objective of this research is to initiate cooperative and collaborative work between the automotive industry, traffic control device industry and infrastructure owner-operators in a partnership to improve the potential for TCD LED display detection and comprehension via the ADS machine vision (camera or other sensor) interfaces, facilitate information processing by the ADS and support the safe deployment of ADS on the nation’s transportation network. The major tasks or activities of the work should include:

Phase I: ● Identification of all known relevant material, equipment and process constraints of automated vehicle sensors and LED based automotive and traffic control device equipment including a global assessment of automotive and electronic equipment state of the practice. ● Identification of motion blur, sensor sampling speed (camera shutter speed or similar), vehicle speed, and ADS data processing function interdependencies. ● Identification of all related and affected standards such as NEMA TS 4 and TS 10 or Institute of Transportation Engineers (ITE) signal specifications. ● Identification of all components that must be modified or considered when attempting to resolve the sensor and LED equipment pairing such as LED refresh rate, sensor sampling speed, and data processing. ● Development of a work plan to identify economical and feasible equipment standards proposals that address the ADS sensor and LED equipment pairing issue. ● Include in the work plan, an outreach effort to coalesce the various industries around the proposed standards and facilitate the ability of ADS sensors to detect and read LED TCD and automotive displays.

Phase II: ● Execution of the work plan o Establish sensor sampling speed range acceptable from a motion blur and image capture perspective (e.g., Nyquist frequency). o Establish LED pulse width modulation (PWM) cycle range that will ensure multiple cycle completions during the sensor sampling range. o If all LED TCD and automotive devices can be operated within the PWM cycle range, then information to be shared with industry standards groups else begin exploration of alternative solution methods such as: ▪ multiple image integration. ▪ localized signal to indicate when LED image will be available for sensor read (established time to “on-state”) ▪ other….. ● Outreach plans to include: o AASHTO, USDOT FHWA and State Department of Transportations o National Committee on Uniform Traffic Control Devices (NCUTCD) o ITE (Signal Specifications) and NEMA (TS 4 and TS 10 standards) o International Municipal Signal Association (IMSA) o Society of Automotive Engineers (SAE) o and American Society for Testing and Materials (ASTM)

---

Submitted By

Adam Hopps

National Operations Center of Excellence

202-680-0091

Attachments