Automotive Vision Industry Research (Part 2): Visual Perception Enters Deep Water, Algorithms Determine Winners

Zosi Automobile R&D released the “2019-2020 Automotive Vision Industry Chain Research Report (Part Two) Binocular and Others”. According to Sunny Optical, automotive cameras are divided into perception cameras and image cameras.

Perception cameras are used for active safety and need to capture images accurately, and are generally used for forward and interior vision. Video cameras are used for passive safety and store or send the captured images to the user, typically for surround and rear views. Therefore, perception cameras and video cameras are completely different in terms of image quality requirements and temperature reliability requirements.

Perception cameras are used for lane detection, signal light detection, road sign recognition, in-vehicle monitoring, etc. If there is any error in the image captured by the perception camera, it will cause software calculation errors and lead to inevitable consequences. Therefore, the price sensitivity of perceptual cameras is relatively low. Video cameras are very sensitive to price, so there is a fierce price war, the performance requirements are not very high, and domestic companies are involved more, but most of them are not profitable. It can be seen from the 2019 China passenger car surround view front-mounted market pattern that the market shares of various manufacturers in the surround view market are relatively scattered, unlike the front-view monocular market where the market is relatively concentrated, with TOP6 accounting for more than 90% of the market.

Automotive Vision Industry Research (Part 2): Visual Perception Enters Deep Water, Algorithms Determine Winners

Source: Zosi Automotive Research

The front-view camera requires complex algorithms and chips. The unit price of the front-view single camera is close to 1,000 yuan, and the binocular cost is more than 1,000 yuan. The unit price of rear view, side view and built-in camera is around 200 yuan.

Breakthrough in binocular field

Domestic start-ups have made breakthroughs in the binocular field, especially in the commercial and special vehicle markets. Zhongke Huiyan Binocular has shipped more than 10,000 units, which are mainly used in Apollo series unmanned vehicles, Jiangling light buses, sanitation vehicles, unmanned boats, tractors, patrol vehicles, etc. The typical applications of the binocular products of Zhongke Huiyan are AEBS and height limit detection. About more than 30 city bus groups are equipped with Zhongke Huiyan binocular system. Binoculars are used for the detection of height-limiting devices, which is the first creation of Zhongke Huiyan. In addition to the conventional height-limiting poles, various non-standard height-limiting devices such as small arches and culverts in the countryside can accurately perceive and timely alarm through sound and light. The device alerts the driver. RVs, special vehicles, etc. all have a strong demand for limited and high detection.

Automotive Vision Industry Research (Part 2): Visual Perception Enters Deep Water, Algorithms Determine Winners

Source: Zhongke Wisdom

In 2019, Hammerhead Shark announced the development of a free binocular system, which can use two completely independent monocular cameras to achieve a binocular system to get rid of the inherent defects of binocular equipment, which is large in size, complex in process, difficult in installation, and high in cost.

Automotive Vision Industry Research (Part 2): Visual Perception Enters Deep Water, Algorithms Determine Winners

Source: Hammerhead Technology

The two cameras are installed independently and can be fixed on the right side of the vehicle body. There is no need for a rigid connection in the middle, and there is no need to strictly control the angle and spacing of the cameras. The core principle of “Free Binocular” is self-calibration technology. Even if the camera is slightly deformed and moved during use, Hammerhead’s algorithm will automatically detect and recalibrate, eliminating the need for regular binoculars. Regular recalibration Calibration work. In April 2020, Foresight announced a partnership with FLIR to integrate FLIR’s infrared cameras, which combine technologies from visible light stereo vision and thermal imaging stereo vision, to provide accurate obstacle detection in harsh light and weather conditions. The data fusion between the two stereo channels can effectively solve the non-reporting or false-reporting in extreme cases such as tunnel entrances with rapid changes in brightness and darkness.

The millimeter-wave report released by Zoth pointed out that millimeter-wave radar is eroding the territory of other sensors. The same goes for cameras. Many companies use monocular cameras for ranging and 3D imaging, trying to replace binoculars or lidars. For example MAXIEYE. MAXIEYE’s first-generation IFVS-200 series is based on machine learning solutions, and the third-generation IFVS-500 series is based on deep learning solutions to achieve monocular ranging and 3D scanning. The IFVS-500 series enables monocular vision products to perform 3D scanning like lidar, and can also scan 3D scene point clouds close to lidar within 50 meters, provide direct ranging function of targets, and realize vehicle range of 200 meters. Detection, pedestrian and small target obstacle detection in the range of 100 meters.

Enhance the visual ability in extreme scenes

Both inside and outside the car, the need to gain vision in poor light or even darkness means utilizing infrared technology. ON semiconductor‘s RGB-IR image sensor uses NIR (near-infrared) technology, while another manufacturer, Trieye, uses a short-wave infrared (SWIR) camera. SWIR cameras have the advantage of being able to see objects in any weather/light conditions and can identify road hazards (eg icy roads) in advance.

Automotive Vision Industry Research (Part 2): Visual Perception Enters Deep Water, Algorithms Determine Winners

In April 2020, Alibaba DAMO Academy developed an ISP processor for in-vehicle cameras. According to the results of the road test, using the Dharma Academy ISP processor, the vehicle camera’s image object detection and recognition ability in night scenes is more than 10% higher than that of the mainstream processors in the industry, and the originally blurred annotations can also be clear. identify. On May 19, 2020, OmniVision officially released the image sensor OX03A2S equipped with Nyxel near-infrared technology. This 2.5-megapixel ASIL-B rated sensor is designed for external imaging applications and can be used in low-light or even no-light environments within 2 meters of the vehicle body. The OX03A2S is capable of detecting and identifying objects in low-light environments that other image sensors cannot.

Automotive Vision Industry Research (Part 2): Visual Perception Enters Deep Water, Algorithms Determine Winners

Source: OmniVision

Visual perception enters deep water, algorithm decides the winner

With the large increase of visual sensors in smart cars, the addition of different types of visual sensors, the amount of data generated, brings challenges and opportunities to algorithm processing. In the visual perception system, Mobileye, the leader of the visual ADAS algorithm, uses a variety of independent perception algorithms to achieve redundant superposition. The purpose is to improve the accuracy and stability of perception in the two dimensions of Detection and Measurement. Detection is to determine what object is perceived, and Measurement is to obtain the 3D information of the perceived object by calculating the 2D picture of the camera. In the Detection dimension, Mobileye uses 6 independent algorithms:

3D Vehicle Detection (3DVD): Identify target vehicles in 2D images and label them with 3D bounding boxes.

Full Image Detection: Mainly used to identify large objects (such as passenger cars or trucks) at close range on both sides of the vehicle.

Top View FS: Focus on identifying and labeling unoccupied roads in the screen.

Features Detection (eg Wheels): Focus on identifying objects with unique features, such as wheels.

VIDAR: Generate a 3D image through triangulation of multiple cameras, and then import the 3D image into the lidar perception algorithm for object recognition.

Scene Segmentation (NSS): Through pixel-level recognition, different types of objects are segmented and marked with different colors.

Source: Mobileye

Tesla is also a leader in visual perception algorithms, and Tesla calls its deep learning network HydraNet. The entire HydraNet contains 48 different neural networks. Through these 48 neural networks, 1000 different prediction vectors can be output. In theory, HydraNet can detect 1000 kinds of objects at the same time. To enhance its algorithmic capabilities, Tesla specifically acquired computer vision startup DeepScale. In order to catch up with Tesla and Mobileye in visual perception algorithms, OEMs and Tier1s are expanding their teams of software engineers. Algorithm ability will become one of the decisive factors affecting the performance of visual perception.

The Links:   CM165YE4-12F LJ320U27 BSM300GB120DLC