Design and Implementation of Vehicle Pedestrian Detection System Based on FPGA

The vehicle pedestrian detection system is a kind of solution to car driver assistance. The system can detect pedestrian in dangerous and send early warning to drivers automatically. Therefore, it has practical significance to develop a robust、real-time and stability pedestrian detection system which can reduce and avoid pedestrian accidents effectively. This article completes the vehicle pedestrian detection system based on FPGA. It used the cyclone II EP2C70 DSP development board provided by the Altera Corporation. By testing, the system can determine a size of 320 × 240 grayscale image takes about 723ms at the clock frequency in the 100MHZ. It achieves the desired functionality. The system has better real-time and reliability. At the same time, it has small size, easy to control and the most important is that it has broad application prospects.


System Function
The main task of the vehicle pedestrian detection system based on FPGA is to process the images which captured by the video cameras and identify pedestrian in the video by using of the Cyclone II EP2C70 DSP development board [1]. Simultaneously, it satisfies the requisitions of robustness, timeliness and stability.
The system includes the following five parts: image acquisition, ROI segmentation, human feature learning, pedestrian detection, processing the resulting output.
Firstly, we design and implementation of pedestrian detection algorithm on PC. Secondly, propose hardware implementation according to the effects of algorithm. Finally, transplant pedestrian detection module according to the overall design of the system structure and achieve pedestrian detection system based on FPGA

Hardware of the system
The hardware of this system is mainly used in the Quartus II integrated Nios II design tool -SOPC Builder. Firstly [2], complete the construction using the SOPC Builder system module. Secondly, complete a comprehensive system used Quartus II and compile into a hardware configuration file. Finally, generate configuration file downloaded to the target device, and complete the pedestrian detection system.
The image acquisition part used to drive the CMOS image sensor and transform image data which acquainted by the image sensor from Bayer to RGB, and stored in the SDRAM memory. The image acquisition part of the structure is shown in Figure 1. The functional image acquisition can be divided into I 2 C sensor modules, CMOS sensor data acquisition module and the format conversion module.
The SDRAM memory is large and access time is short, so using the SDRAM for image sensors and display cache. SDRAM controller stored the data which acquainted by the image sensor in the SDRAM through the four ports (two read ports and two write ports), and then display this image data on the monitor, as the CMOS image sensor and VGA monitor cache.
During the operation in the SDRAM, address pins, data and control signals of the SDRAM are valid in a certain time window. SDRAM clock must be switched to capture the correct value at the effective time window. When the clock frequency is low, the clock naturally falls within the valid window; when the frequency is higher, it must compensate for SDRAM clock, so that to align it with the effective window. After several debugging, The SDRAM which with 100MHz clock frequency and delay 3ns data can correctly reading and writing. Image display part of the structure is shown in Figure 2. The function of Image display part is storing the image data in SDRAM and then displayed on the VGA monitor. The image display includes VGA controller, SDRAM memory and 4-port SDRAM controller. In the Provided Terasic TRDB-DC2 package, there is a similar image display driver. The image shows part is designed according to the routine.
Because stored images and display the images' size is different，inevitably lead to the data storage and display abnormal if let the image data stored in SDRAM and output to VGA monitor directly. The SDRAM's clock is 100MHz, while the VGA's the output clock is 25MHz, the image data must be stored in SDRAM, So the system create a FIFO for reading the data of SDRAM and deal with the data by asynchronous clock and synchronous transmission. Reading the image data which stored in SDRAM and sending it to the display control module through the 4-port SDRAM controller. The main task of the display control module is complete the relevant data which is the each component of RGB data for 10 binary data (Low five full complement 0), and add the appropriate synchronization signals for the monitor for display processing.

Part of system software
System software development process is as follows: Firstly, the system should be initialized. Secondly, let it find the characteristics of the human body. Thirdly, divide the image of the region of interest [3]. Finally, based on the training model, it can determine whether it is a human being. If it does, then output the human's frame. For the circumstance when three consecutive images come out to be humans, we can result in red light, or green light.
System initialization. The system initialization includes the main image sensor system initialization and the VGA display initialization. The image sensor initialization includes initializing CMOS and initializing the sequence (register address, the value of sequence) through the I2C bus.

Mechatronics and Information Technology
The VGA display initialization includes setting the image data buffer for the received image, configuring the image display resolution, opening VGA controller and closing the VGA controller interrupt.
ROI Segmentation. The ROI segmentation is a possible alternative pedestrian area. It plays a very important role in the design of the entire system [4]. To identify the regions of interest which extracted by the image as a input of the pedestrian recognition module, so that reducing the number of scans to do pedestrian recognition and improving the rate of pedestrian detection system greatly. This paper extracts the region which may be including some person based on the shape of the image characteristics. Mainly used in the Sobel operator to solve the level of image edges and vertical edges [5].
In the area which can be filmed by the monocular camera in front of the vehicle, the driver can only care the pedestrian in about 5m and 1.5m away. So process the 2/3 of the half of the image and the middle of the width of 2 / 3 parts of the image which captured by the camera. We believe that the feet which have obvious level characteristics will appear in the images of the lower half according to the feature. Therefore, the horizontal edge just processes the half of the image.
Region of interest segmentation includes some steps. Firstly, we find the bottom border of the pedestrian candidate area and find out the strong edge position using the Sobel horizontal templates. The level property of the feet's location is very strong, so we can think the location which with strong horizontal edge is the part of feet. Secondly, process the image and extract the vertical edges of the image using Sobel vertical templates. Thirdly, seek symmetry of the part with the vertical edge features. Finally, determine the location of the border regions of interest based on human aspect ratio (5:1).

1) Pre-processing module
The main task of the data preprocessing module is reduce the input image from the size of 640 × 480 to the size of 320 × 240, and convert it to grayscale. On the one hand, reduce the occupancy of the FPGA resources. On the other hand, raise a follow-up of the image processing speed.
Image data is RGB 16-bit data types and 14 to 10 bits of data is the R, 9 to 5 bits of data is the G, 4 to 0 bits of data is the B. The procession for extracting the R, G, B component of point a at the current frame is as follows: r = a>>10; g = (a&(0x03e0)>>5); b = a&0x001f; The calculation of typing the RGB image into gray image is To improve performance and reduce the computational complexity, so change the decimal arithmetic to the score computation. The operation formula becomes: Image scaling uses a simple interpolation method, which can extract the image pixel evenly. In this paper we can form a new image includes odd rows and odd columns and sores the data in the320 × 240 of the frame [240] [320]. Figure 3 is the procession of extracting a horizontal edge of image. Fig. 3. Flow chart of extracting horizontal edge T is the threshold which assured by Otsu method. Through the software debugging, we can find an average value of the threshold is 120. So in this paper we use the value of threshold is 120 directly. The request method of Gx is:

3) Sobel vertical edge extraction
Find the vertical edges of the image and seek the position of symmetrical objects in the image based on vertical edge image. Procession is shown in Figure 4. Gy's Method is The method of extracting Vertical edge image and horizontal edges are basically the same. The different is calculation template.

4) Solve the region of interest area
Based on processing results using Sobel horizontal edge extraction module and vertical edge of the module, we can determine the location of the object with stronger symmetry in the image (area of leg) by using the aspect ratio for 2:1. Merge pixel border which is less than 5. Finally, we can get ROI.
The chance of pedestrian recognition algorithm is core of the system. It directly determines the accuracy of pedestrian recognition and real-time. In this paper we detect pedestrian by using the characteristics of improved gradient direction combined with SVM classifier and obtain better recognition affection. We detect pedestrians when the camera is fixed. In low light situations, parts of shot pedestrian detection results are in Figure 5 (a) and Figure 5 (b) shows. The number of pedestrians is small and the background of the target area is not complicated, so the recognition of pedestrian is better and there is no case of false detections and missed situations.
Traffic conditions, the detection results of some test shots on the streets with more pedestrian are in Figure 5 (c) and Figure 5 (d) below. System can correctly identify the majority of pedestrians when testing environment includes complex background and more pedestrian, but there have some missed and false detection of the condition. Compared with the fixed camera static, recognition performance did not decline. The reason is that the system for processing each frame and the camera movement are not necessarily linked.

Advanced Engineering Forum Vols. 2-3
The time for region of interest segmentation is 264ms.The time for calculating Integral Figure is 47.07ms. The time for each region of interest is 0.3ms and ROI category is 19.7ms when extracting features. Recognition rate for pedestrian is 89.3% and false recognition rate is 6.3%.