Compile? |? James
Editor? |? Han Xiao
Apple has begun to provide new ideas for the automotive industry.
Since June, Apple has obtained 20 new patents in the automotive field, covering three aspects: autonomous driving, intelligent cockpit and intelligent body.
In this newly exposed patent, Apple's brain hole is also wide open.
Is its use related to Google Alpha? Take AI-like technology to teach machines to drive, and use long-wave infrared cameras and other visual sensors to improve the efficiency of lidar; ? Turn the windshield of the car into an AR-HUD screen, automatically number the buildings outside the car to facilitate navigation, introduce your own FaceTime video call function, and even make a drowning escape system for the car …
As the technology company with the highest market value in the world and the influential IT giant in the field of consumer electronics, every time Apple sets foot in a field, it will bring a lot of amazing innovations and push the product experience to a new height.
Although I have given up my Titan car-making plan and turned to autonomous driving. However, in recent years, Apple has been applying for patents related to various smart cars. Through these documents, we can not only see the innovative technology and all kinds of whimsy that Apple has brought to the industry, but also see how powerful the technology reserve of this technology giant is.
Apple obtained a patent for automobiles this year.
In June this year, Chedong System reported dozens of automobile patents obtained by Apple in the first half of this year (please refer to "Decrypting 70 Patents of Apple Cars! The brain hole is too big for Tesla to catch up with), this article mainly interprets the new patents obtained after June.
First, teach the machine to drive with AI? Google Alpha? Using the same technology
In about five months, Apple * * * obtained 10 patents for autonomous driving, covering three major areas of autonomous driving: sensors, communication and control.
Nowadays, with the development of autonomous driving for many years, the computing power of computing devices has been greatly improved, but how to control autonomous vehicles is still a big problem. The difficulty lies in how to make timely and reasonable decisions for self-driving cars. Moreover, the more vehicles on the road, the more complicated the traffic environment and the more difficult the decision-making.
In Apple's patent map, Apple describes that self-driving vehicles can complete driving actions such as automatically entering and leaving the ramp, automatically changing lanes and overtaking. In order to complete these actions, the autopilot computer needs to complete two actions:
1, predict the coming traffic conditions, and make various plans for the vehicle trajectory;
2. Choose a correct driving action from various schemes.
This decision-making process is very similar to that of playing chess, so Apple adopted the decision-making process with Google Alpha? Go is a method similar to machine learning, that is, Monte Carlo tree search, which has been widely used in board games and open games.
When the vehicle enters the automatic driving state, the vehicle actions within a certain time range can be planned according to the traffic conditions, including acceleration, deceleration, lane change and so on. These actions can be generated by Monte Carlo search tree.
Four steps of Monte Carlo tree search
A cycle of Monte Carlo tree search includes four steps, namely, selection, expansion, simulation and back propagation. Starting from the root node r, the child nodes are gradually deduced downward. Each child node is the state of the vehicle, and the edge connecting the child node and the parent node represents the action of the vehicle.
For example, the vehicle can choose the decision from the root node R to the child node L. If it succeeds, it can continue to expand from L to E. After reaching the node E, it will continue to make decisions by random selection and enter the simulation process. After this automatic driving process, the system will be deduced from the bottom sub-node to R, so that the machine can continue to learn and become more powerful.
Diagram of Monte Carlo tree search in patent
In practical application, Apple's decision-making process is more complicated. By providing recorded vehicle actions, actions and benefits from simulation for different Monte Carlo tree search models, the Monte Carlo tree search in reinforcement learning strategy neural network and supervised learning value neural network can enhance the breadth and depth respectively.
In fact, the method of controlling autonomous vehicles is not only machine learning, but also stochastic technology and deterministic technology, and the final input to the decision-making process is the mixed calculation results of the three.
To sum up, in the process of motion decision-making, autonomous vehicles may encounter random and dynamic environment, infinite space, potential huge vision and real-time limitations. These factors lead to the self-driving car can not drive freely on the road, but rely on the calculation of the self-driving computer.
Second, the sensors are together? Lidar can see more clearly.
Self-driving vehicles are also inseparable from various sensor devices. For high-level autonomous driving, lidar is even more essential. Apple first obtained several patents on lidar on 20 18. Early patents show that Apple will use lidar for ranging and use the data for autonomous driving.
Since June this year, in less than half a year, Apple has obtained three patents related to lidar again, and further elaborated how to use lidar.
In a patent called "Detecting Objects and Ranging by Remote Sensing", Apple uses sensor-assisted lidar to achieve more accurate ranging and identification.
In the process of automatic driving, not all road traffic conditions need the attention of vehicles, but some important traffic conditions need more attention. Nowadays, visual sensors are all over the car body, and there are almost no blind spots. These sensors can detect the traffic around the vehicle in real time.
Infrared camera assisted lidar ranging
At this time, the visual sensor can determine a key attention area, and the laser radar can add ranging to make the distance judgment more accurate.
In another patent entitled "System and Method for Solving Hidden Objects in Vision", Apple uses a long-wavelength infrared sensor (LWIR) to assist lidar, which can detect animals and pedestrians that may appear on the road in extremely dark light.
With the help of long-wave infrared rays, the thermal energy of objects can be detected with high resolution in a certain area, and objects that may not be detected by other sensors can be detected.
Both patents show that lidar can be used in combination with other sensors. For example, lidar and infrared camera can measure the distance at the same time, which can improve the accuracy of ranging. Long-wave infrared sensors can be combined with monochromatic sensors, RGB sensors, lidar and other sensors to enhance the target detection effect.
At the same time, the patent document also shows other sensor components, including camera, biological detection processor, lidar, vertical cavity surface emitting laser pulse controller and so on. These sensors can participate in the detection of objects during autonomous driving.
Interestingly, the first Apple device equipped with lidar was not a car, but this year's iPad? Pro and iPhone? 12? Pro series, the lidar module installed on mobile devices can scan the 3D information of a room and improve the focusing speed in low light.
Images seen by iPhone lidar
It is reasonable to imagine that Apple's current layout on lidar hardware may also be applied to autonomous driving.
Third, the windshield becomes a big screen? You can also use FaceTime at will.
In addition to a large number of autonomous driving patents, Apple has not slacked off in-vehicle technology, putting excellent solutions in mobile devices into cars.
AR augmented reality has always been one of Apple's key technologies. Now, if you browse Apple official website, all its main products can be viewed through AR, so you can have a more accurate understanding of the size and color of the equipment before buying.
View iPhone with AR
At the same time, a large number of AR applications in Apple App? The store offers even Angry Birds, the AR version of My World.
In the automotive industry, there is a lot of discussion about AR this year, mainly focusing on AR-HUD.
In September this year, the 1 1 generation S-class sedan released by Mercedes-Benz was equipped with AR-HUD. Its display area is equivalent to a 77-inch monitor. According to the introduction of Mercedes-Benz, this AR-HUD adopts the imaging unit of digital micromirror devices used in cinema projection, which consists of 6.5438+0.3 million microlenses and a high-resolution imaging matrix. The light source is efficient.
Display effect of Mercedes-Benz S-Class AR HUD
When driving, virtual information such as animated arrows and vehicle distance can be superimposed on the real world within the range of 10 meter to realize the ar effect.
Apple, which has been in the layout for many years, naturally hopes that AR technology will enter the car, and the main application is AR-HUD. According to statistics, so far, Apple has obtained four patents on AR-HUD.
In a patent named "Augmented Reality Display System" obtained by Apple, the functions that Apple hopes AR-HUD can realize are described in detail.
1, providing navigation information and translating street signs.
The original intention of HUD is to let the driver concentrate on driving and try not to let his eyes leave the road. AR-HUD is a great improvement on the basis of HUD, which extends the original small-area and fixed-content HUD to a larger area and provides an augmented reality experience based on the external environment.
Based on this goal, the easiest thing to think of is to move the navigation information from the car or instrument to the windshield. Through specific signs, drivers can get better navigation services, and at the same time, more drivers can make navigation "completely shut up".
AR-HUD translatable street sign
AR-HUD only provides navigation service, which is relatively simple, so Apple has added the function of translating street signs. If you drive overseas, the road signs in different countries are not uniform and the language is unreasonable, which may lead to serious safety hazards. If vehicles can recognize street signs in real time, it will be convenient for more drivers to convert the meaning and text information of street signs into easy-to-understand content.
2. Is there a FaceTime call on the windshield? The co-pilot can also watch the video.
FaceTime calls were defined by Jobs as Apple once again defined the iPhone, so FaceTime calls are essential in the car.
Through AR-HUD, the driver can speak directly on the windshield, and the display screen can be as big as a real person, which makes the video call more telepresence.
However, during driving, FaceTime mobile phone will not block the driver's line of sight, the co-pilot can see the complete picture, and the main driver can only make audio calls. When the vehicle speed drops or stops, the FaceTime screen will be displayed on the driver's side.
FaceTime video phone can be realized through the windshield.
In fact, after FaceTime was introduced into the car, the windshield was a big screen. During the trip, the co-pilot can watch videos and even play games through the windshield.
In fact, the application of AR-HUD is not only that. Apple described another use of AR-HUD in another patent called "System and Method for Identifying Objects in Mixed Reality Interface".
In the complicated city, when we walk to the blocks with high buildings, we can't say the names of the high buildings and don't know how to describe them, which has become a big problem for people to travel.
Apple marks high-rise buildings or shops facing the street in AR-HUD. Select the number in the car, and the car will automatically drive to a specific location, which is convenient for the occupants in the car to reach a specific location.
Four, four windows are all smart glass? And a drowning escape system.
In addition to software and services, Apple has also made great efforts in hardware. In terms of fuselage structure, Apple also hopes to be "fully armed" and bring the hardware industry to a new height.
According to statistics, Apple * * obtained 10 patents on car bodies within half a year. It may be that Apple attaches great importance to privacy protection and applied for many patents related to "color-changing windows". Since the beginning of this year, seven patents have been obtained. At the same time, the protection of car body safety has always been valued by Apple. Even if the vehicle falls into the water, it can also make correct operation, which is convenient for the evacuation of people inside the vehicle.
1, the smart window can be virtual or real? 5-layer components protect privacy
Apple wants to design the window into multiple layers, and each layer uses different structures to adjust the overall color atmosphere and transparency of the window. This can not only ensure that the occupants in the car are not harmed by harmful light, but also protect their privacy without sticking a film.
In a patent called "Windows and Systems", Apple hopes to control the absorption of light by using electrically adjustable windows, so as to achieve the effect of controlling the light in the car. This technology can be used not only for side windows, but also for windshields, skylights and rear windows, and can adjust the light transmittance of the whole window.
According to the description in the patent document, the window glass is actually very similar to the liquid crystal display module. It can also be used to display images and adjust colors. For example, practical functions such as displaying navigation information and providing internal lighting can be realized on the window glass.
In another patent called "Window with Heatable Infrared Transparent Area", Apple added a filter layer on the window to block infrared light. This can avoid excessive heat accumulation when the car is exposed to the sun. In addition, the rear window of the vehicle will be equipped with an ohmic wire, which can be heated and defrosted after being electrified.
In Apple's latest patent, "System and method for dynamic privacy protection and window coloring" describes the layering of windows in more detail.
From the patent drawing, it can be found that the car window consists of polycarbonate layer, conductive layer and guest-host liquid crystal layer respectively, in which the polycarbonate layer and conductive layer wrap the guest-host liquid crystal layer inside.
The window assembly * * * has five layers of display assemblies.
Through this component, the window can dynamically adjust the window color. If there is strong light, the window will adjust the glare. If the visibility outside is low, the windows will be clearer and brighter. In addition, the occupants in the car can also manually adjust the window coloring to protect the privacy inside the car. Vehicles can even use lidar devices. If someone outside the car peeks into the car, the vehicle will adjust the transparency of the window to prevent the privacy inside the car from leaking.
At the same time, in order to prevent the invisible light of the camera, laser radar, night vision device and other components from being absorbed by the window interlayer, Apple has reserved a certain area on the window for these hardware to ensure that the invisible light can smoothly look far through the window.
2. Is it convenient to escape after drowning? Seat belts, doors and windows are the key.
Apple also obtained a patent called "Dynamic Element Protection Measures". After drowning, the vehicle can make a correct response according to the drowning situation to help the occupants in the vehicle escape as soon as possible.
It is very dangerous for a vehicle to drown.
When the vehicle is partially submerged in water, one or more computer systems in the vehicle begin to intervene, and operate to close the window, lock the door, unlock the safety measures, turn on the floating equipment, send out a distress signal and adjust the seat position, so as to facilitate the occupants in the vehicle to escape. At the same time, remind the passengers in the car to prepare for escape.
Vehicles will detect life in real time. Using high-resolution UWB radar equipment can detect the possible vital signs in the car, including breathing and heartbeat detection, and even find the pets left in the car in time.
After it is determined that all the occupants in the vehicle have escaped, the vehicle will enter a state of self-protection and lock the doors and windows to avoid further losses.
In addition to windows and safety patents, Apple also obtained a patent for intelligent suspension, which made the vehicle run more smoothly. At the same time, the ventilation duct can be hidden in the frame of the car body, which makes the interior space more abundant. As can be seen from these details, Apple makes cars with great care.
Conclusion: "Apple Smart Car" is worth learning from the industry.
At present, Apple has obtained more than 100 automobile patents. The patents obtained in the past six months show that Apple's emphasis on autonomous driving is rapidly increasing, which also makes Apple's autonomous driving route gradually clear.
Nowadays, in the smart car industry, whether in vehicle intelligence, autonomous driving or three-electric system, Tesla and other new car-making forces are a beautiful landscape in the automobile industry. Giants like Apple are constantly moving in the automobile industry, and they have also given some excellent lessons to the new forces of making cars. How to make a car like Apple? Can the car defined by Apple still be welcomed by the market?
This article comes from car home, the author of the car manufacturer, and does not represent car home's position.