Virtual Reality (VR) Live Streaming and 360° Video: A Primer

Have you heard about virtual reality (VR)? What about 360° Video?

The tech world is abuzz about all things VR. Maybe you’ve even seen people wearing strange cardboard boxes strapped to the front of their face. Don’t worry, this isn’t some strange parasitic alien creature. It’s most likely Google Cardboard, a VR platform that turns a smartphone into an immersive digital environment.

Virtual reality and its partner, 360° video, are both in their infancy. These technologies have been gestating for decades, but only now is computing power and bandwidth catching up to make them a reality. It’s likely that we’ll see a renaissance in VR video over the coming years.

In this article, we’re going to examine the connections between live video streaming and the virtual reality revolution. First, we’ll introduce the technology and go over history, current implementations, and the future. We’ll explain some technical differences, and then discuss VR live streaming. Finally, we’ll share a VR workflow and discuss challenges with this new tech.

After reading this article, you should have a solid footing in VR technology. Sound good? Let’s get down to business!

A brief introduction to virtual reality

The idea of virtual reality is simple: create a synthetic environment that people can modify and enter at will. If you’ve seen The Matrix, you know the drill. VR is all about an immersive experience that attempts to mimic reality, or create sci-fi or fantasy realities.

 

Past

VR is a relatively simple idea, and it’s been around for a long time. The history of VR goes back to the 1930s, when science fiction author Stanley G. Weinbaum first came up with the concept.

Over the next 70 years, VR technology advanced very slowly. The first VR machines weren’t digital at all. Instead, they were large boxes you stuck your head into that used mechanical means to change artwork around you.

early-history-vr-and-360-videoMany people may also be familiar with the “View-Master” and other stereoscopic simulators that used small round cards slotted into a headset to project images toward the eye.

Between the 1980s-2000s, VR became more common in niche applications. Sega and Nintendo developed some of the first video game VR systems. Military and aviation industries spent large sums to develop VR flight and combat simulators for training pilots and fighters. NASA developed a similar platform to pilot the Mars rovers. Medical firms developed several training platforms for surgery simulations.

Present

Since the turn of the millennium, VR has taken off. In 2010, the first prototype of the Oculus Rift was released. The Oculus VR corporation was subsequently purchased by Facebook in 2014 for $2 billion. It took until March 2016 before a product version of Oculus Rift hit the market. The platform features an OLED display at 1080×1200 pixel resolution per eye and a 90 Hz refresh rate.

Oculus has inspired a number of copycats, including products from Sony and Google. The PlayStation 4 platform now supports VR gaming via the PlayStation VR headset, launched in October 2016. Google’s affordable Cardboard headset (~$15) takes a different approach, replacing a built-in screen with a slotted-in smartphone. Several thousand apps and games on the Google Play store now support VR, and web content can also take advantage of the platform.

Increasingly, more companies are getting in on the action. In November 2016, the NFL partnered with YouTube and Google to create a 9-art series filmed in 360° VR. The NBA is now broadcasting several full games per year in 360° virtual reality.

 

Future

As processing power and bandwidth increase further, VR is going to become even more mainstream. It’s completely feasible to assume that VR content will be the dominant medium for multimedia within the next decade or so.

This change will affect news media, entertainment, sports, television, music, video gaming, and dozens of other industries. Businesses, conferences, and religious groups will also find increasing use for VR video. Smartphones are the defining technology for the current generation. VR could very well be the defining tech for the next.

Stereoscopic content is also part of the future for VR. This basically means that each eye sees a slightly different image. This is similar to 3D video. Instead of a single flat viewing surface, you either have a screen for each eye or a special set of glasses that filters out certain types of light. This allows for real depth perception in a virtual environment. Some special cameras with dual lenses already support stereoscopic video recording.

What’s the difference between virtual reality and 360° video?

person-using-a-vr-headset-for-videoLet’s discuss an issue of terminology. Technically, there is a big difference between virtual reality and 360° video. Virtual reality, in full implementation, allows you to move around in an artificially-generated environment. You can interact with objects, walk, run, climb, and otherwise function as you would in the real world.

Modern VR implementations have limitations that don’t allow them to reach this ideal. Notably, if you’re wearing a headset and try running, you’re likely to run into a wall or end up in the middle of the street. The best most VR games can do is allow you to use a video-game controller to actually move around in the virtual world. You can swivel and turn in place as much as you want, but advanced movement isn’t possible yet.

Most of what passes for VR these days is actually 360° video. This simply means video that’s been recorded in every direction at once. This allows users wearing a headset to “look” around. For users inside a circular arrangement of monitors, they can simply turn around.

However, in 360° video, you can’t change the position of the cameras. That’s determined by the person who made the recording. That’s the difference between true VR and 360° video. Some people describe 360° video as the “bridge” content—standard flat video stretched across a circular frame.

The closest current bypass of this limitation is in some military simulators. These advanced VR devices place the user on an omnidirectional treadmill, and sometimes in a harness as well. This combination can simulate freefall, movement in any direction, and so on.

We’re still a ways out from a fully 100% VR system that truly allows body-based movement and feels like real life, but we’re getting there.

VR live streaming: the next frontier

In the world of video, development happened in stages. First, there were video recordings, then television broadcasts, and then online video and live streaming.

VR is following a similar path at an accelerate rate. Although VR video is still a niche technology, live streaming is already possible. The Verge calls live VR video “the gateway drug between what most people watch today and the immersive era of virtual reality that is just getting underway.”

(Technically speaking, all VR live streaming is really 360° video at this time.)

Streaming 360° live video content is possible right now. In fact, it’s already happening. If you have a fast enough internet connection, you can try to find some 360° live streams on YouTube today.

However, most viewers and content creators aren’t prepared to make or watch this content. People need better equipment and more bandwidth than is typically available. We’ll discuss these challenges more later in this article.

In the near future, those obstacles will be disappearing. Internet speeds are increasing, computer power is rising, and compression algorithms are getting more effective. These factors, combined with the fact that VR streaming will be a truly powerful communication medium, make the transition inevitable. It’s coming soon.

Walkthrough of a typical VR video streaming workflow

vr-and-360-video-status-and-futureLet’s look at the different parts of a VR or 360° video streaming workflow. There are a few differences here when compared with a traditional broadcast setup.

Rig and studio

First up is the camera itself. Here’s the thing: you need more than one.

In the past, 360° video recording was extremely expensive, requiring custom-built rigs of multiple cameras linked to a single controller.

However, that’s changed over the last few years. Modern cameras that allow 360° recording are surprisingly affordable. In fact, you can get your hands on one for just a couple hundred dollars. Broadcast-quality 360° cameras that will capture ultra high resolutions can be had for four or five thousand dollars.

Audio is another important part of VR video. As we’ve written in the past, audio is the most important thing for audiences. A gorgeous 360° recording can be completely ruined if the audio sounds off.

In real life, turning your head changes the orientation of your ears and alters how things sound. For this reason, VR and 360° video have to be combined with spacialized, stereo sound recording for maximum impact. That means omnidirectional microphones and careful mixing to ensure the right sounds are reaching the viewer at the right time.

All this will require support (like tripods) and mounting hardware. This may translate into a completely different studio setup. In most traditional video recording situations, the camera sits in an alcove and mostly points in one direction. What’s in front of the camera has to look nice; what’s behind it doesn’t really matter.

That’s no longer the case with 360° video. This brings up new challenges regarding set design, camera placement, cable management, and even where to put camera operators and assistants. It’s a whole new ballgame!

 

Stitching

After recording, 360° video usually has to be stitched. That’s because almost all 360° cameras are really multiple cameras recording separate video streams. Some cameras skip this step.

There is a variety of software out there for stitching these different sources into one spherical video. It’s possible to stitch using software like Adobe’s Premiere Pro, but dedicated programs like VideoStitch Vahana VR and Kolor Autopano will often produce better results.

VideoStitch Vahana VR is also the only software available at the moment that is capable of stitching VR video in real-time for live streaming. With several video cards, it is possible to capture video streams from multiple cameras, stitch them together into a 360° video, and output that video via RTMP to your streaming services provider.

 

Encoding

Of course, any streaming video requires encoding. This is just as true for VR and 360° video. After video is stitched, it goes to encoding. This can take place in software or dedicated hardware encoders.

However, this type of video content tends to have extremely large file sizes. That means that encoding is particularly important. File sizes need to be minimized. Ideally, streams should be delivered at a number of different bit rates to cater to viewers with different internet speeds.

Additionally, more processing power will be required to encode this video content. A more powerful computer or a higher-performing hardware encoder may be necessary.

Upload and Delivery

Issues with file size also become important when it comes to upload. Here at DaCast, we recommend that your internet upload speed be at least 2-5 Mbps (Megabits per second) in order to broadcast high definition video.

However, VR video will usually require additional data. Instead of sending a single 1080 x 1920 pixel video stream, you may be sending much more data. Currently, most 360° videos on the web use 4K resolution (3840 x 2160 pixels). That’s four times more data than 1080p HD.

Additionally, VR video usually looks better at higher frame rates. Altogether, you’re looking at 4 times the data or more. These days, most good-quality 360° or VR video is encoded at around 20-30 Mbps. That’s a good trade-off that produces acceptable quality with the h.264 codec. However, people will increasingly be using advanced codecs that will make file sizes smaller, reducing bandwidth requirements.

In the future, 8K video resolution (7680 x 4320 pixels) will become the norm for VR video.

 

Viewing

Next comes the most important step: viewing. Your audience can only access VR live streams with extremely robust internet connections and powerful devices. For an immersive experience, viewers need a headset like Oculus, PlayStation VR, or the HTC Vive. On mobile or desktop, they’ll also need a video player that supports navigation around a 360° frame.

Challenges in 360° and VR video streaming

bandwidth-and-data-challenges-360-vr-video-live-streamingThe biggest challenges in VR video have to do with data. Simply put, it takes a massive amount of data to depict a 360° field of view.

Most high-quality live streams max out at 1080p resolution. That’s generally not enough for VR or 360° video. Many more pixels are needed.

For example, let’s say that your field of view using a VR headset is 120°. That means that to show 4K video resolution wherever you look, you need to record in 12K resolution! That’s a pixel size of 12288 x 6144, more than 75 megapixels.

VR content also benefits from very high frame rates. Most VR video has to record in at least 90 frames per second to appear truly no different from life. That requires even more data.

Another VR challenge is stitching. Currently, combining video from multiple cameras is difficult. Software makes it easier, but problems can arise. Artifacts and blending issues can occur where frames overlap. Sometimes it doesn’t look quite right.

On the bright side, these challenges are diminishing bit by bit. Improvements in internet speed, processing power, video compression, and software usability will make it easier to produce and view VR content in the near future.

Conclusion

It’s an exciting time for VR and 360° video live streaming. The technology is starting to mature. We expect to see a lot more of this type of content in the near future. Right now, many live streaming providers offer some level of support for 360° video. DaCast, for example, supports 360° streaming as long as your chosen video player can support the format.

Want to try us for free?
Enjoy a 30 days free trial to discover our platform and its video features.

untitled

Leave a Reply

Your email address will not be published. Required fields are marked *