Virtual images emerge in a real-world environment before a user's eyes, creating the sense that the CG images actually exist. Mixed reality (MR) is an imaging technology that seamlessly merges the real and virtual worlds in real time. Canon's MR System allows users to interact with full-scale, realistic CG images from any point of view, an effect difficult to achieve using existing video systems. The MR System creates a realistic imaging world that allows users to enjoy an entirely new visual experience.
How does the MR System create this imaging experience?
- (1) Two video cameras inside the head-mounted display (HMD), one each located in front of the left and right eyes, capture video from the real world, which is then sent to a computer.
- (2) After gauging the user's positioning and orientation based on registration markers captured by the video cameras and data from the built-in sensors, the system accurately aligns the real-world and virtual images and displays them on the small monitors located inside of the HMD.
- (3) The system's free-form prism, comprising cutting-edge optical technologies, enlarges the video displayed on the small monitors to enable users to experience high-impact three-dimensional images with low distortion.
The MR System is configured using Canon proprietary imaging technologies.
One of the challenges faced during the development of the MR System was minimizing the size of the head-mounted display (HMD). This was overcome through the development of the free-form prism.
The free-form prism, which is designed using a free-form surface shaped neither like a flat plane nor a sphere, enlarges the merged images displayed inside the HMD. The images shown on the display are refracted and reflected within the prism, helping to not only enlarge the images, but also deliver a clear image across the entire field of view without distortion or blur caused by optical aberrations. The free-form prism is also compact enough to substantially reduce both the size and weight of the HMD. Canon's proprietary optical technologies have made the development of the free-form prism possible.
Another challenge faced during the development of the MR System was the realization of accurate alignment technology that enables the depiction of CG images in real-time at a specific location in the real-world environment based on the user's position and orientation. For example, when a CG object is rendered on top of a table, it may sink into the table or float above it if the real-world and virtual images are not aligned properly. The mixed reality world will appear unnatural if the MR System does not correct for changes in alignment.
Drawing from the company's image-processing technologies cultivated over many years, Canon overcame this issue by developing a high-precision, high-speed marker detection method. The video cameras employed in the HMD capture and read registration markers to calculate the exact position and orientation of the HMD. In addition, internal gyro sensors and optical sensors deliver further alignment accuracy. When creating a mixed reality environment, the MR System's dedicated calibration tool realizes greatly enhanced setting and calibration operational efficiency compared with previous methods, which were cumbersome and complicated.
This technological breakthrough has enabled substantial progress to be made in MR technology.
Canon began research and development for the MR System in 1997. The company has since been accumulating hardware and software engineering capabilities to achieve the practical application of mixed reality in society.
As product lifecycles grow progressively shorter, it has become necessary for the manufacturing industry to introduce products to the market in a timely manner. When used during the product design phase, the MR System allows users to view full-scale CG images that are instantly responsive to their position and orientation, enabling the evaluation of product design and usability. The system can also help to reduce the number of prototypes required and, in addition to shortening the amount of time spent on development, can contribute to minimizing costs and environmental impact.
The MR System can be used not only in the design and manufacturing sectors, but also in a wide range of industries and fields. Satisfied with the functionality and quality offered by the MR System, Canon launched the system in July 2012.
Canon keeps trying to further improve the MR System to make it more convenient for users and to expand the ways it enriches people's lives. The company is working on reducing the size and weight of the HMD, widening its angle of view and enhancing the image quality of merged visuals. Regarding alignment, technological advancements have enabled a reduction in the number of registration markers required, and progress is being made in creating a marker-less system.
As the MR System realizes further advancements, its scope of application will expand into a wide range of fields. For example, the system can support the medical industry by visualizing the state of diseases and treatments for patients, and can act as a presentation tool for new home and car showrooms. In the future, there will be more opportunities for the general public to experience the MR System.
With the potential to open up a new world of imaging possibilities, Canon's MR System takes advantage of the rich visual information of the real world and the flexibility offered by CG images, creating new possibilities and value across every industry and sector.
Canon is working toward the day when the MR System, as one of the company's core technologies, promotes innovation in society.