For ordinary users, they may not be deeply aware of this change, but for the global lidar market, this may be a message that indicates the future direction.
Lidar is not a new technology. It was introduced in 1980s and commercialized in 1990s. In the past decade, autonomous driving has risen. Lidar is regarded as the "eye" of autonomous driving and has attracted much attention.
However, the car-mounted lidar is expensive and bulky, and the price may exceed 10,000 yuan, which will greatly increase the cost of the car, so that elon musk, the founder of Tesla, said that "people who use lidar are idiots", but at present, the car-mounted lidar can only be used for autonomous driving test and cannot be commercialized on a large scale.
Apple's lidar technology is not so complicated, but Apple has had several precedents to promote the whole industry chain by itself, and this change may also happen to lidar.
Apple is good at pushing mature technologies to the mass consumer market, and lidar is another case.
Behind the iPad lidar is still ToF (Time of Flight) three-dimensional vision technology. The essence of ToF technology is to use radar to emit a variety of light waves, which are reflected by obstacles. By measuring the flight time between the emitted light and the reflected light, the depth can be estimated and the distance can be determined, so that the 3D image of the measured object can be obtained. In the 1960s, with the invention of laser, it became a reality to test the distance by using light waves.
Since 20 16, mobile phone manufacturers have set off a wave of "everything can be ToF". Huawei, Samsung, OPPO and vivo smart phones are keen on remote control of the screen, ToF Mito and other functions (optimizing the depth of field function of mobile phone photography).
Is this iPad pro cooking cold rice?
That's not true. ToF technology can be divided into iToF and dToF. According to insiders, the principle of iToF is relatively simple, and it is difficult to integrate with infrared light source scanning. However, its shortcomings are also obvious. First, the accuracy is low and the anti-interference ability is poor. Second, its power consumption is high, which greatly increases with the increase of distance. This means that iToF can't capture high-precision moving 3D images, and its role in practical application is quite weak.
In contrast, dToF improves the measurement accuracy through SPAD (single photon avalanche diode) +TDC (time-to-digital conversion circuit) with faster response time, and has lower power consumption and stronger anti-interference ability. However, dToF has high hardware requirements, high integration difficulty and high cost, and is rarely used in consumer electronics products.
Apple chose dToF technology. According to its introduction, a special lidar scanner uses direct time of flight (dToF) to measure the light reflected at the farthest distance of five meters in indoor or outdoor environment. It can detect at photon level and run at nanosecond speed. Not only that, the lidar scanner cooperates with the rear dual cameras, motion sensors and the architecture in iPadOS to measure the depth.
The application bottleneck of dToF technology in small consumer electronic products is that SPAD is difficult to be miniaturized. Eric Zhang, a user in Zhihu, said that the bigger the SPAD, the higher the resolution of lidar. He thinks that the lidar of iPad Pro is a breakthrough in this respect, and it is estimated that the resolution of lidar can reach between QVGA and VGA (320 * 240 pixels to 640 * 480 pixels). In other words, the camera performance of the iPad Pro will be improved, and the accuracy and fluency of the AR experience will be greatly improved.
For the AR effect on the new iPad Pro, Apple officially said; "Augmented reality applications can give people a stronger sense of reality: placing virtual objects can now be done instantly; The realistic object occlusion function allows virtual objects to be inserted back and forth in the real scene structure; Motion capture and person occlusion functions have also been optimized for higher accuracy. "
In addition, the blessing of the super wide-angle camera can also make the shooting angle of the iPad Pro 2020 larger, and it is also a greater improvement in 4K video and images.
With the help of lidar, iPad Pro 2020 adds more imagination to augmented reality (ar) and other fields. In this regard, Apple gave three typical scenarios: AR height measurement, AR games and IKEA AR home decoration. Taking the AR game "Hot Magma" as an example, iPad Pro's lidar scanner can turn the living room into a magma field, and children need to jump on the furniture to avoid the simulated magma on the floor.
Many mobile phone manufacturers love ToF technology. Why was Apple the first to land a lidar scanner?
This stems from Apple's early idea of building a car. In 20 14, the "Titan Plan" of Apple's car-making plan was officially born. At that time, Apple hoped to replicate the success of the iPhone, produce complete driverless cars from scratch, and compete with other automakers' projects. To this end, Apple recruited soldiers everywhere, and the number of department personnel was as high as 2,700.
However, Apple's road to building a car is not smooth. After the change of the person in charge and the adjustment of the research and development direction, the Titan Project was shelved, and Apple turned to focus on the driverless system in 20 16. In 20 19, according to foreign media reports, Apple said in an email to the California Employment Development Department that its self-driving car department planned to lay off 190 employees.
Despite stumbling on the autopilot track, Apple still has some technological breakthroughs. 2018110/October, a new patent of apple depth camera named "Multi-range TOF sensing" was officially announced by the US Patent and Trademark Office, which mainly focused on the optimization of photoelectric elements in the camera, especially the lidar sensor.
After TriLumina developed a VCSEL vertical cavity surface emitting laser with high beam quality, low production cost and low power, Apple applied for a new patent, with the theme of combining lidar with VCSEL technology and applying it to new products such as ToF camera and ar.
2065438+In April, 2009, it was reported in Reuters, and people familiar with the matter were quoted as saying that Apple was looking for laser radar equipment that was smaller, cheaper and easier to mass-produce than the current technology. These patents and hardware may have greatly promoted the birth of iPad Pro equipped with lidar.
The iPad pro lidar, which originated from making cars, is naturally very different from the car lidar.
Mark, co-founder of RoboSense and chief operating officer of lidar in the field of autonomous driving, claimed in an interview with Zhidx that "this is completely different". The lidar in a man's wallet-sized car and the lidar on iPad Pro can only recognize several meters, and the detection standards are different, and the cost is lower. Comparatively speaking, vehicle-mounted lidar mainly ensures safety, long recognition distance and high performance requirements.
Algorithm engineer, a supplier of L4 driverless trucks in Tucson, told Future Auto Daily (ID: auto-time): "Compared with ordinary passenger cars, trucks need to transport goods, so they can't brake suddenly, so the perceived distance of self-driving trucks needs1000 m."
Mark especially speculated that the lidar on the back of the new iPad Pro is likely to adopt a non-scanning flash scheme. This method has the problem of short perceived distance in in-vehicle applications, and it is just right in the scene set by Apple. In addition, according to Mark, compared with the vehicle-mounted lidar, the dToF scheme of iPad Pro is a fixed planar 3D sensor, and the latter is a rotating single line or multi-line.
However, even if there is no breakthrough innovation in technology, the annual sales of iPad Pro and subsequent iPhone are huge, which will inevitably make the whole industrial chain of lidar mature rapidly, and at the same time drive the application of lidar in other electronic consumer goods and promote the popularization of lidar technology. By then, manufacturers can also let go of their hands and feet to achieve more technological innovation.
Author | Jiao
Editor | Xu Yang