There are fewer and fewer highlights of high-end smart phones, and Ray Tu Tu has been unable to show the advantages of mobile phones. So a more domineering dual camera appeared. The dual cameras mentioned here are not like the previous smart phones, where one camera is in front and one camera is in the back, but a pair of eyes with two rear cameras to simulate people's eyes, so as to achieve more camera functions and better camera effects.
Generally speaking, the camera interface of smart phone is MIPI (Mobile Industrial Processor Interface) interface. In the past, there were only two MIPI interfaces on the mobile phone platform, one for the front camera and the other for the rear camera. To be a dual camera, the platform needs to support at least three MIPI interfaces. In fact, in the previous high-end platform, in order to achieve higher pixels, dual ISP(Image? Signal? Processing, image signal processor), for example, in order to support the 16M camera, two 8M-capable ISPs will be used. This platform is likely to have only two MIPI, but this does not prevent engineers from doing front single camera+rear dual camera.
Dual camera application
This is the problem. What can dual cameras do?
1, dual cameras can measure the distance and can be used for distance-related applications.
It is easy for the human eye to locate the distance of an object, but when one of the eyes is closed, the positioning ability will drop a lot. Dual cameras are applications that simulate human eyes. To put it simply, measuring the distance is to calculate the angles θ 1 and θ2 between the object and the left/right camera through an algorithm, and add a fixed Y value (that is, the center distance between the two cameras), so it is very easy to calculate the Z value (that is, the distance between the object and the camera).
However, it is also easy to calculate. If the center distance between the two cameras is too small, the computable object distance will be very close. If you want to calculate a longer distance, you must widen the distance between the left and right cameras.
As shown in the above figure, because the distance of an object can be judged by the algorithm of dual cameras, it is easy to make some special effects through this feature, such as:
Background blur
One of the most prominent features of SLR cameras is the large aperture. Because the dual cameras can measure the distance of different objects, aim at objects that need large aperture and blur other objects with different distances, the effect of large aperture can be easily realized.
)
B, background replacement
Because the distance can be measured, the subject in the object can be extracted, and the background can be changed, it is easier to carry out matting than PS.
C, background special effects
Because the distance can be measured and the theme and background can be separated, it is easy to do anything with the background, so I won't describe it too much here.
D, measuring distance
This picture clearly identifies the distance of different objects, and this distance information is identified with different colors. When AP obtains the distance information of different objects, it can realize the above functions.
2, dual cameras can do optical zoom.
Optical zoom mainly means that the left and right cameras use different FOV (angle of view), which makes the vision of the two cameras different. When users need wide-angle photos, they use a left camera with a viewing angle of 85 degrees to get a wide-angle effect. When users need telephoto photos, use a suitable camera with a viewing angle of 45 degrees to obtain telephoto effect.
In order to make the objects photographed by the left and right cameras overlap highly, the dual camera module of optical zoom should not be too far away from the applied camera module, but the left and right cameras should be placed as close as possible.
If the FOV of the two cameras is different, one is large and the other is small, and then the effect between the two optical lenses is realized by algorithm, optical zoom can be easily realized.
If you don't use dual cameras, the text will be unclear after you enlarge the picture.
If you use dual cameras, the text will still be clear after enlarging the picture.
This picture is a combination of a wide-angle picture and a telephoto picture, and the photo of the intermediate posture is calculated by an algorithm, so that the details are not distorted.
3, the dark light effect is enhanced
Generally speaking, dark light enhancement is done in two cameras: a standard camera with RGBG and a black-and-white camera with RGBG filter removed. RGBG is used to obtain the color of the object, while black-and-white camera is used to obtain better light input to judge the light intensity of the object. Then the two images are fused to obtain better dark light enhancement.
At present, there are two fusion methods:
Taking the black and white picture as the main body, the color of each pixel obtained from the color picture is pasted on the black and white picture, and the two pictures are fused.
Taking color pictures as the main body, the light intensity of each image obtained by black and white pictures is compensated to the color pictures, and the two pictures are fused.
As for which method is more suitable for integration, different people may have different views, so I won't discuss it here.
Similarly, for dark light enhancement, in order to make the objects photographed by the left and right cameras overlap highly, this dual camera module is also required to be as close as possible.
It should be noted that Huawei P9 actually chose the module of this mode.
Of course, some insiders said that the effect of this algorithm is not obvious at present. Dark light compensation is really helpful to users, especially when shooting at night. However, some customers think that Sony and Samsung's dual PD technology is very good, and they are more willing to use dual PD cameras for dark light compensation.
Dual cameras or dual PD have good dark light compensation effect. You can compare Huawei P9 and Samsung's Galaxy S7 edge, and you will have the answer.
This usually uses a color+black and white camera. The brightness intensity of the picture is obtained by a black-and-white camera to compensate for the dark light of the picture.
4.3D shooting and 3D modeling
In fact, the algorithms of 3D shooting and 3D modeling are similar to those of distance application, but they require higher accuracy, and sometimes even infrared ranging is needed to judge the distance more accurately.
When it comes to the algorithm of dual cameras, you have to mention ISP(Image? Signal? Processing image signal processor), ISP's main function is to post-process the signal output by the front-end image sensor, and its main functions include linear correction, denoising, removing bad points, interpolation, white balance, automatic exposure control and so on. Only by relying on ISP can the details of the scene be well restored under different optical conditions, and ISP technology largely determines the imaging quality of mobile phones.
In the era of functional computers, ISPs are all built on cameras, and cameras with different pixels are matched with ISPs with different performances. With the increasing pixel size of mobile phone camera, the requirements for ISP performance are getting higher and higher. If the ISP is integrated into the camera sensor, the camera module will be too large, which will even affect the photographing effect. Therefore, in the era of smart phones, ISPs are generally on the main chip SoC. In order to achieve better results, some brand customers even add an ISP at any cost to achieve better and more professional photo effects.
A good camera algorithm needs a good ISP. ISP and algorithm complement each other and are indispensable. Dual cameras have higher requirements for ISP performance. First of all, in order to process the signals of the left and right cameras at the same time, a single ISP can no longer meet the needs of dual cameras. This requires a dual ISP to achieve this function.
Taking dark light enhancement as an example, color/black-and-white images enter their respective ISP channels and calibration channels, and then the two images are matched (for example, the same part of the two images is extracted and the part shot by only one camera is removed), and then the related images are processed by algorithms such as occlusion, detection and compensation. Finally, the two images are fused to enhance the color. Of course, in fact, ISP does more things with algorithms than this picture.
Of course, there is also an episode. After all, there are two ISPs, and the two ISPs have some problems in processing speed and processing ability. In order to ensure that two ISPs can sample at the same time, it is necessary that the pictures taken by dual cameras are taken at the same time. One solution is to let the sensor have a synchronization signal pin. The synchronization signals of the two cameras are connected, and each time a picture is read, it will be time stamped. ISP ensures that the photos taken by the left and right cameras are taken at the same time through time stamps, and finally merge.
Different from the shooting of general 3D movies. The two cameras on the mobile phone can't produce enough visual difference in the process of image shooting, because the distance between the two cameras is different from that of the human eye. And in order to make people get a more obvious 3D visual effect. So enhancement often requires algorithms.
Because the distance can be measured, the subsequent dual cameras can not only realize 3D photography, but also carry out 3D modeling. At this time, I think the importance of dual cameras is even more important.
Other effects have also been enhanced, such as HDR and resolution improvement. These functions can actually be realized by a single camera, but dual cameras can make the effect better, so I won't list them one by one.
Summary:
At present, these functions are the most common functions of dual-camera phones. Background blur/replacement, dark light effect brings more photo effects to users; Optical zoom makes us feel the camera function of zoom function; But personally, the most exciting thing in the future is the 3D function.
This year VR is so hot, where does the VR material come from? Still have to rely on the optimization of the dual camera algorithm. If the algorithm of 3D photographing and modeling is mature, dual cameras will be more popular.
Dual-camera ecological chain
Earlier, we talked about the application and principle of dual cameras. Now let's pay attention to the industrial chain of dual cameras.
1, dual camera algorithm supplier
Because the algorithm needs to cooperate with ISP, the algorithm and ISP complement each other. If you want to do the algorithm well, you must have a good ISP.
As a major platform provider, Qualcomm/MediaTek has its own ISP, so it has also developed its own dual camera algorithm. As for its algorithm, it has yet to be tested by the market.
As sensor suppliers, Sony, Samsung and OV are also actively developing dual-camera algorithms, and have not seen mass production products yet. However, in the era of functional machines, there is no ISP. These camera sensor suppliers have to match their own ISP when making 2M/5M, so they all have their own experience in developing ISP. Therefore, the development of dual-camera algorithm is more experienced.
Apple acquired Linx last year, and it also has patents and algorithms for multi-cameras. Whether it will be used on its own dual-camera model depends on this year's iPhone7. In theory, Linx has enough algorithms, and Apple will not go to a big factory to buy IP licenses for dual cameras.
In addition to these platforms, sensor vendors and branded smartphones will develop their own dual camera algorithms, and we will list other vendors one by one:
Huajing, 1996 was established in Taiwan Province province, mainly researching and developing independent ISP chips. Some high-end mobile phones, cameras and cars all use the shell of their ISP. Dual-camera phones also have their own ISP. Distance application, optical zoom and dark light compensation have all made achievements.
An Israeli company. The advantages of the algorithm mainly lie in optical zoom and dark light compensation. In the field of depth, there are also some studies. From the point of media propaganda, hTC has adopted its algorithm.
Founded in 1994, ArcSoft is headquartered in the United States and has technical centers in Shanghai, Hangzhou and South Kyoto. The strengths of ArcSoft are optical zoom and dark light compensation. The second optical zoom image and dark light complement image in the previous article are from Irisoft.
Shanghai Xingxinwei is a company founded on 20 1 1, mainly engaged in the research and development of image processors. At present, its products are mainly used in the vehicle market. As one of the few companies that develop independent ISPs, they are also developing dual-camera algorithms and ISPs. X-Chip will be a dark horse with great potential after the double-shot market.
Dual cameras have just come out, so each family has its own advantages and disadvantages in algorithm. However, judging from the effect of the dual-camera mobile phones that have been launched at present, the algorithms of each family need to be improved. After all kinds of algorithm capabilities go up, it is bound to make dual cameras become the standard of mobile phones.
2. Dual camera sensor supplier
Because two cameras need to synchronize time stamps when taking pictures, this requires the camera sensors to have synchronous signals. At present, the sensor suppliers that provide this synchronization signal are Sony, Samsung, OV and Gekewei. So these products are mainly used for dual-camera sensors.
3. Dual camera module supplier
At present, there are many module factories that can make dual cameras, including Guangbao, Yu Shun, Xinli, Namgya, O-Film, Samsung Electromechanical, Qiuti and so on. However, the main companies with mass production experience are Guangbao, Yu Shun and Sony. Huawei's models mainly use modules from Guangbao and Yu Shun. And O-Film and Samsung Electromechanical, relying on their strong factory capabilities, are now making great strides in the field of dual camera modules. Namuga maintains a good communication relationship with various algorithm companies, is a supplier of Samsung mobile phones, and is gradually making efforts in the field of dual camera modules.
However, different functions require different modules. Let's continue to take the four functions mentioned above as examples:
Distance application, the general practice, is the size of the camera, common camera specifications such as 13M+2M, 13M+5M.
Through these two ways, we can better calculate the distance from the object to the lens.
Optical zoom dual camera module, the most important thing is that the two cameras need to have different FOV, similar to the following figure:
Different FOV, different focus, and then through the algorithm, the optical zoom function can be realized.
Generally, dark light enhancement is shot with RGB full-color camera and monochrome black-and-white camera, as shown in the following figure:
In terms of algorithm, the amount of light entering is mainly read by the black and white camera on the right to compensate the color of the RGB camera on the left.
3D shooting and modeling, this way is somewhat similar to the distance correlation module. It's just that 3D requires higher accuracy of distance. In this case, it is more necessary to put the distance between the two cameras a little farther, and some even use infrared-assisted positioning to achieve ranging, and finally realize the role of 3D shooting and modeling.
Difficulties in manufacturing dual camera modules;
Generally, there are two kinds of dual cameras: * * substrate or * * * bracket. As shown in the figure below:
In the case of a * * * substrate, two camera sensors are placed on the same substrate, and then an FPC is led out from the substrate. If the bracket is used, as shown in the above figure, the sensors are fixed by the bracket, and each sensor has its own substrate and FPC.
* * * Advantages of the bottom plate: Two sensors can sit on the same flat plate and are resistant to falling.
* * * Disadvantages of the substrate: low yield, resulting in high price.
* * * Advantages of the bracket: high output and good price.
* * * Disadvantages of the bracket: Because it is two independent sensor modules, it needs AA calibration to make them on the same plane, which is difficult and has poor fall resistance.
In short, the two methods have their own advantages and disadvantages. At present, only Huawei adopts the * * * substrate method. But either way, the current rate of return is very poor, so the cost is still very high.
The future of dual cameras
If we improve the algorithm and improve the module yield, dual cameras still have many advantages worth doing.
However, because different functions have different requirements for module placement, dual cameras may not meet everyone's requirements for cameras at present.
When Google launched Project Tango, it skillfully put forward a concept of three cameras: when ranging and 3D modeling are needed, two cameras with a long distance can be used; If you do optical zoom and dark light compensation, you can use two close-range lenses. Even this sense of depth can be combined with infrared rays to measure the distance more accurately.
In any case, with the evolution of dual camera algorithm and the increase of VR demand, cameras play an increasingly important role. Mobile phones with dual cameras, even front dual cameras and rear three cameras, are likely to become more and more common in the next year.