The Minds Behind the Magic : Special Interview / Merging 3-D computer graphics with real space MR(Mixed Reality) Technology

part 1 At the Core of HMD is a Freeform Prism

I tried out Canon's HMD (Head Mounted Display), and it is truly like wearing a pair of magic goggles! Once you put it on, a three-dimensional virtual car appears in the space right front of you, and you can actually sit in the driver's seat. First, tell us about the basic construction of this "magic."
The key device for MR (mixed reality) technology is the HMD, which incorporates a CCD video camcorder that captures video images from real space, and a display device that overlays computer graphics (CG) images onto the captured real video images. The display itself is compact and offers ultra-high-definition output, so the user isn't really aware of it. The freeform prisms enlarge the images to make them visible.

Overview of MR TechnologyOverview of MR Technology

So, using freeform prisms means that the images are more or less the same size as when seen with the naked eye?
That's right. The images from the small display are enlarged using the same principle as a magnifying glass. Using a three-sided prism, the surface on one side is used for both total reflection and light transmission, causing the images to be refracted and reflected four times.

Optical Configuration of Conventional Mirror-Type Displays

HMD optical system used for MR

So, this means you're taking advantage of Canon's specialty — optical technology?
That's right. Thanks to Canon's extensive optical design know-how and internal production technologies, products are manufactured according to the design specifications.
Why do you call [the prism] "freeform"?
This is because the surfaces on all three sides are neither flat, nor are they a sphere with a central axis. It's a special shape that ensures that the viewed image is displayed clearly and not distorted or blurred at the edges due to aberrations in the optical system. Part of the surface includes an undulated curved surface that makes appropriate corrections for any aberrations.
Have you done anything special to the camera that captures the images?


Yes. A camera is positioned in front of each the left and right eyes so that the image from the right-hand camera is shown on the right-hand display, and the image from the left-hand camera is shown on the left. So, in the HMD, we've aligned the center of the image capture by cameras with the center of the user's field of vision. This is how we create the feeling of spatial depth for the user.
How much can actually be seen with the HMD?
Unfortunately, at the moment only about 30 percent of the human field of vision is viewable. The wider the field of vision gets, the more that the peripheral images suffer from distortion and image resolution deteriorates. You know, this is the most difficult point when designing a freeform prism.
We've incorporated an electrical correction system to deal with distortion, but it's extremely difficult to try to expand the display range while maintaining a clear image.
Electrical correction? Even though the image reaches the viewer's eyes through the prism?
We created a circuit that deliberately applies distortion to the display in the opposite direction from the distortion coming from the prism.
So the distortion is cancelled out?
Yes. The crucial point is for the image to look natural to the viewer, so that it doesn't matter if it is distorted on the display.
In order to have a clear image, it must be important to be able to deal with electrostatic noise.
Yes. There are many places where noise can come from, such as the CCD image sensors themselves, the analog circuitry, the metal housing of the HMD, the cables that connect the HMD and the PC, the power lines, and so on. So, we use double and triple circuits as noise countermeasures in order to prevent image problems and protect against malfunctions.
Canon Technology

Top of Page