Seven augmented reality optical schemes and their advantages and disadvantages
1, direct or off-axis reflection
This technology is similar to a projector, which can directly project images onto glasses, such as Glass Up, that is, a mini projector is installed on the right temple, and the lens is used as a reflector, which will project an area of 320×240 on the lens, and form parallel light after reflection to enter the human eye for imaging. Although this scheme can't multitask like mobile phones and tablets, it is enough to handle some short messages and simple charts. Meta glasses also uses a similar technology-off-axis mirror, which is different from the standard parabolic mirror. It can directly guide and focus the incident parallel light at a specific angle and support infinite focusing. The extremely compact projector is hidden in the frame, and the images on the translucent LCD are projected onto the beam splitter by the LED light source for imaging, thus providing stereoscopic vision.
2. Prism optics
The simplest one is a 45-degree prism, which reflects the light generated by the display from the glasses frame to the human eye, and at the same time lets the light in the real world pass through. This is simple and cheap, and the well-known Google Glass has adopted this scheme. However, due to technical limitations, even the shortcomings of Google Glass are obvious. The angle of view is only about 20 (the prism method can only be made thicker to enlarge the FOV), and the light needs to pass through the semi-reflective semi-permeable membrane twice, resulting in a low utilization rate of light energy (about 20%) and a dark picture. Limited by the manufacturing process, the lens is thick, the large-area lens is expensive and the yield is low.
3. Free-form surface prism type
Free-form surface refers to a surface whose surface shape can not be processed continuously, which has the arbitrary characteristics of traditional processing and forming, and its design difficulty is much higher than the first two categories. Generally speaking, its shape is wedge-shaped glass, which is a rotationally symmetric XY polynomial free-form surface. In this structure, the light is transformed by a prism to form a virtual magnified image, and the color difference and distortion can be eliminated by the emergent surface of the freeform surface and the reflecting surface of the freeform surface, and the imaging quality is clearer, and the viewing angle can reach 54 degrees, which can be further improved by using the double freeform surface prism. The disadvantage is its large volume, about 7- 10mm thick.
4. Optical waveguide+holographic technology
The waveguide can achieve a field of view angle of 30-40 within 3 mm on the glass, which is thinner than ordinary myopia glasses and can be used in combination with ordinary glasses. However, its design is also the most difficult. The basic principle of this technology is total reflection and diffraction of light. As shown in the figure, the holographic waveguide helmet display system is mainly composed of microdisplay, holographic grating and flat waveguide. The image becomes parallel light after passing through the micro-collimating lens, and enters the optical waveguide to reach the first holographic grating. Due to the diffraction effect of holographic grating, parallel light changes its transmission direction to meet the total reflection condition, and propagates forward along the waveguide direction without loss. When the parallel light propagates to the second holographic grating, the total reflection condition is destroyed, so that the parallel light exits from the holographic waveguide and enters the human eye for imaging. Due to the existence of holographic waveguide, optical images can be vertically deflected and propagated. This not only reduces the propagation distance, but also keeps the center of gravity of the optical system in the head. At the same time, the use of folding mirrors is reduced, which is beneficial to the simplification and lightweight design of optical systems. However, this technology intelligently realizes monochrome display. In order to achieve the color effect, three layers of lenses must be used to project three primary colors, namely red, green and blue, and different lights are combined to form colors. At present, HoloLens adopts this scheme.
5, optical waveguide+reflection technology
Because of diffraction effect, holographic grating scheme will lead to dispersion and image blur. Therefore, Lumus, an Israeli company, adopted a light-guiding optical element (LOE) device, which adopted a simpler multi-reflection layer structure instead of a holographic grating, as shown in the figure. The principle of LOE device is similar to periscope, but multiple mirrors are used to enlarge the exit pupil. Each mirror reflects parallel light, and these mirrors are on the same image. Lumus' representative product, PD- 18, has a resolution of 800×600, a viewing angle of 26× 20, an exit pupil 10mm and an exit pupil distance of 23 mm ... The device has a thickness of 2.3mm, a weight of less than 70g, a brightness of 1200fL, and a display area transmittance of 70%.
6. Light field technology
As another major technical route of near-eye 3D, light field technology is represented by Magic Leap. The biggest advantage of this technology is that it allows users to focus freely, and looking far and near will not produce the fuzzy feeling like traditional photos or videos. The technical core of this method is the optical fiber projector. Magic Leap is based on the principle that the output direction is tangent to the optical fiber when the laser propagates in the optical fiber and exits from the optical fiber port. By changing the shape of the optical fiber in three dimensions, especially the tangent direction at the optical fiber port, the direction of laser emission is controlled and directly projected on the retina.
7. Video overlay technology
The above scheme is relatively difficult, and the video overlay technology can realize the AR effect more simply. Video overlay technology directly uses a camera instead of a lens to observe the real world, and overlays virtual things on the scene shot by the camera. This method avoids the difficulties in optical design and can enhance people's visual experience. For example, by using an infrared camera, we can see the scene of infrared spectrum. But it also brings another problem, that is, the amount of data operation is huge and needs to be solved urgently.
Four "bottom" AR implementation methods
If the above is a professional technical solution, then what we want to introduce below is an "amateur" solution specially prepared for AR technology enthusiasts.
1, Opencv and C++
The general principle is that OpenCV can identify and locate the mark, and then overlay the virtual object under the camera image through OpenGL to realize augmented reality. The specific idea is to use SIFT algorithm to identify (extract feature points and describe them with feature vectors, and then match the feature vectors of the current view with those of the target object), and get the change matrix according to the matching relationship between the identified original target and the frame image, so as to display the 3D object (drawn by OpenGL) and realize tracking.
2、Python
Python is the most elegant language in the world. At present, most computer vision projects are implemented in Python. Of course, using Python CV library can also easily achieve the AR effect. To realize AR effect with Python, we need to apply two open source toolkits, PyGame and PyOpenGL. PyGame is a very popular game development toolkit, which can handle display windows, input devices, events and so on very simply. In the process of implementation, it is necessary to obtain the camera matrix and convert it into OpenGL format, estimate the attitude with planes and marks, and then place virtual objects in the image to realize augmented reality.
3、AR+SLAM
SLAM is mainly used for map reconstruction. In AR, SLAM algorithm is usually used to obtain camera attitude. There are some SLAM development resources and AR-SLAM project cases on the internet, which can be downloaded for reference if you are interested.
4、ARToolkit
ARToolKit is a library written in C/C++ language. For developing AR programs, the most difficult part is to cover the virtual image to the user's viewport in real time and align it accurately with the objects in the real world. ARToolKit uses image technology to calculate the relative position between the camera and the tag card so that programmers can overlay their virtual objects on the tag card. The fast and accurate tag tracking provided by ARToolKit enables people to quickly develop many newer and more interesting AR programs. The implementation steps are: program initialization-capturing a frame of video input-then detecting the tag card-calculating the transfer matrix of the camera-drawing a virtual object-closing video capture.