CeBIT 2015

Panoramas for your tablet

Research News / 2.3.2015

Most people are familiar with the fictional world of “Star Trek,” in which the characters can use a holodeck to create and interact with virtual worlds. It is possible to recreate a similar effect in the real world using 360-degree panoramic images. Researchers are bringing them now to our tablets – including individual camera work and editing.

© Photo Fraunhofer HHI

At CeBIT, Fraunhofer HHI is showcasing an app that allows users to navigate panoramas using their tablets.

Panoramic video thrusts viewers right into the middle of the action. “Even a 180-degree panorama leaves you really feeling part of the action,” says an enthusiastic Christian Weissig from the Fraunhofer Institute for Telecommunications, Heinrich-Hertz-Institut HHI in Berlin. While the technology has existed for a number of years now, when Germany’s first IMAX cinema in Munich closed in 2010, 3D technology took over in drawing viewers into the action, and it seemed as if panoramic technology had already passed its peak. “Too expensive, not commercially viable” – that was the brutal verdict on the technology that acts as the inspiration for the “Star Trek” holodeck. Thanks to the research done by Weissig and his team, the panorama could soon pop up where we might least expect it: on the screens of smart TVs, smartphones and tablets. “Ultra-HD-Zoom” is a prototype that allows users to select and navigate around high-resolution segments of panoramic images. The researchers will be showcasing their tablet app at the CeBIT computer expo in Hanover, March 16-20 (Hall 9, booth E40).

Video panoramas are created by combining the images recorded by a series of high-resolution cameras. Fraunhofer HHI’s OmniCam system, for instance, uses 10 HD cameras. This leaves the technology capable of creating 360-degree panoramic images in real-time – making it a fascinating proposition for covering live events. Last year for instance, Fraunhofer researchers recorded the soccer World Cup final between Germany and Argentina in Rio de Janeiro. They have also recorded the concert given by the Berlin Philharmonic on the occasion of the 25th anniversary of the fall of the Berlin Wall. The recordings have a resolution of 2000 x 10,000 pixels. “Sadly, none of us have a panoramic cinema at home, and the devices in our living rooms and in our pockets are simply not capable of processing this amount of data,” explains Weissig.

The cameraman-spectator

What is possible, though, using currently available LTE networks, is to transmit individual segments of the panorama. “What we’ve done is split the panorama into a set number of segments. These segments are made available to each user concurrently, with the app selecting the segments needed to display the desired section of the panorama,” says Weissig. This approach makes it technically feasible for a very large group of people to use a panoramic image at the same time. Of course, they won’t get the panorama at its full resolution, just the individual segments they choose rendered at the resolution of their device. “It’s another step towards personalized television: users taking advantage of the ‘second screen’ to become their own cameraman and take over the footage, maybe by zooming in to a specific point within their chosen segment. Until now, the apps on the market have been able to offer only a selection of static camera angles, or else transmit a full panorama in HD definition,” says Weissig.

Content providers and TV broadcasters also stand to profit, potentially offering the new capabilities as a service of their own. “We are already collaborating with partners who want to implement the technology themselves, for instance to improve the marketing of live concerts,” says Weissig. The investment needed to produce panoramic recordings remains high – but now that cost can be spread across a large number of users via the pricing of the app, which still remains reasonable for each individual user. The technology should be commercially available within the year. “In the meantime, we’ll be polishing the product to improve transfer speeds,” says Weissig.

But isn’t there still a shortage of content? And aren’t panoramic recordings still far too expensive? “The trend is clearly one towards extremely high resolutions – just look at the new 4K TVs or the Japanese broadcasting corporation NHK’s drive towards 8K resolution. Panorama technology is advancing too. In the future, there will be more content and more devices capable of displaying it. The Ultra-HD-Zoom app is a first application that we can expect to be available soon. As such, it’s a pointer as to where panorama technology might go in the future,” says Weissig.

At their CeBIT trade fair booth, the researchers have set up the whole scenario: Visitors can use the app to select a specific camera angle based on live footage. These are displayed as overview images on the right of the screen. If the visitor selects one of the OmniCam cameras, they are free to navigate the content themselves.