With the evolution of automotive technology, Advanced Driver Assistance Systems (ADAS) have become mainstream. The suite of electronic gadgetry that constitutes ADAS is highly impressive in its capabilities and sophistication.
ADAS offers solutions for pedestrian avoidance, traffic sign recognition, lane departure warning, blind spot detection, and much more. Today, one of the striking applications of ADAS is the 360 degree view car camera – a swanky new piece of car tech that can easily turn you into a driving and parking pro!
Many of us would have seen this technology in action in the latest models of cars. For those who haven’t, here are the details.
What is a 360 Degree View Car Camera?
Also referred to as a Bird’s Eye View Car Camera or Surround View Camera System, the 360 Degree View Car Camera is a collection of devices that work together to provide the driver a real-time view of the surroundings of the car. This view is projected directly onto the dashboard hardware, usually the infotainment system.
The surround view camera system software displays the vehicle and its surroundings so that it looks like the car is being filmed by a drone from directly above. The image screen is also split to display the vehicle’s top view on one side of the screen and the front/rear/side view on the other.
One of the biggest advantages of this system is the ease with which drivers can navigate the vehicle into a parking slot, while avoiding contact with nearby obstacles in an astute manner.
While this may seem like wizardry to the uninitiated, in reality, it is just the output of a set of camera sensors and image processors, enhanced with a sprinkling of data science!
Components of an Automotive Surround View Camera System
The 360 degree view car camera system is a synergy of hardware and software that work together and display information related to vehicle surroundings on the infotainment screen. The individual components of this system are as follows:
- Surround View Cameras – 4-6 cameras with wide-angle lenses are integrated in the body panel of the vehicle. Usually, these cameras are placed on the front grille, under the rear-view mirrors on either side, and on the tail. The cameras are located at spots that enable the surveillance of the entire perimeter of the vehicle.
- Proximity Sensors – The proximity sensors are the components that aid in evaluating the distance of nearby objects. These ultrasonic/electromagnetic devices send signals to a nearby object. The time that is taken to reflect these signals will indicate the actual distance between the car and the object.
- Image Processing Software – The data from the cameras are sent to an image processing module. This software takes the individual images and stitches them together to produce a cohesive view of the car and its surroundings in real-time.
- Hardware for Display – The video from the image processing software is projected on an HMI (Human Machine Interface), which is usually the infotainment system screen. Feedback mechanisms such as audio alerts are also used in the surround view camera system to notify the driver of nearby objects.
How Does the Image Processor Create Coherent Composite Images?
Geometric alignment, photometric alignment and composite view synthesis are some of the algorithms used to create composite images in vehicle surround view systems.
The geometric alignment process includes lens distortion correction and transformation of perspective. Photometric alignment process matches the brightness and colour of individual camera views, so that the final image appears to have been captured using a single camera. The composite view synthesis process is where the actual stitching of images take place.
The image stitching process is usually divided into the following activities:
- Image registration – This process includes detection of key points on an image and assigning them to a common ground plane. Custom pattern registration is the image registration process commonly used in 360 degree vision systems.
- Warping – In this step, the undistorted image is deformed to match some defined key points. Techniques such as homography, polynomial deformation or moving least squares are performed for image deformation.
- Blending – This is the step in which the blending and merging of the individual images take place to obtain the stitched image. Some of the commonly used algorithms for the blending step are multiband blending (best resolution), feathering and 50% blending.
Some Crucial Design Decisions for Surround View Camera System
While developing a 360 degree view car camera system, an important design decision is the placement of cameras. It is crucial that the cameras can view the vehicle perimeters clearly. They should also not disrupt the aesthetics of the vehicle or any other functionality. Hence, system designers spend a good amount of time in simulating CAD models of the vehicle to analyse the field of view and blind zones from different angles. This activity is done in multiple vehicle load configurations.
Another aspect to be considered is the process of camera calibration. No two cameras produce the exact same output. If two cameras are mounted at the exact same location, they will still create images that differ slightly. The placement of lens, aging, thermal expansion, etc. are factors that bring about these differences in the camera images. Hence, it is important to perform camera calibration to compensate for these mismatches.
If camera calibration is not executed properly, there could be issues like non-aligned output images and faulty colours. Intrinsic and extrinsic calibration of cameras is crucial for optimum performance of the surround view camera system.
It is also important to design the bird’s eye view car camera system so that it is easy to operate, install and maintain.
Conclusion
360 degree view car camera systems will be an integral part of autonomous vehicles. Today, this technology is not restricted to luxury vehicle models. As motor vehicle regulations around the world evolve to include more safety-related guidelines, ADAS applications such as these will see a significant growth.
