Microsoft officially opened a lab in Zurich dedicated to researching mixed reality technologies and artificial intelligence. The software company is working closely with ETH Zurich. ETH News spoke with the director of the new research and development lab, ETH Professor Marc Pollefeys.
ETH News: Professor Pollefeys, what is mixed reality?
Marc Pollefeys: It’s all about combining the physical world with virtual reality. For instance, getting smart glasses to add virtual objects to the wearer’s field of vision. What’s great about a mixed reality environment is that it lets users interact with the real world and with the virtual objects at the same time.
Can you give us some examples of applications?
Let’s take the example of a technician who has to service a machine. One possibility would be for the machine’s manufacturer to publish a manual and for the service technician to then painstakingly search through it to find the information they need. Mixed reality can guide them through the maintenance work directly on the machine and much more intuitively by showing them each work step “live”. What’s more, today’s mixed reality glasses use cameras to track the exact position and movements of the wearer’s hands. This lets people use their hands to manipulate virtual objects, for instance as a way to remotely control a robot. Another application is navigation within buildings: the service technician’s mixed reality glasses can guide them to a defective fire alarm, and at the same time give them certainty that they are repairing the right one.
You’re leading the new Microsoft lab in Zurich, which is devoted to mixed reality and artificial intelligence. How do the two interact?
Taking someone through a particular task step by step is a good example of that. The computers in today’s mixed reality glasses are not yet very good at knowing which step a user has reached. Artificial intelligence methods can help to determine that. This turns the device into an assistant that helps users do their job.
Credit: “New Microsoft research laboratory collaborates with ETH Zurich”, Fabio Bergamin, ETH Zurich
