Resurgence in XR live sports streaming

A global resurgence is underway in immersive 360 degree (360o) virtual reality or VR live sports video streaming, along with other extended reality (XR) enhancements to fan experiences. This is confronting service providers with new challenges in the battle against the ever-present threat of piracy.

As access to XR live sports streaming becomes available, providers must contend with the threat of piracy. It is a well-known fact that whenever any type of premium services draw big audiences, the video content  becomes a target for the billion dollar global piracy mill. Consequently, they need recourse to content protection mechanisms tuned to the complexities of real time XR services that can cost effectively satisfy security requirements common to licensing conventional live streaming services.

Most significantly, both 3600 and 1800 VR experiences replicating in-person attendance are emerging at the cutting edge of the networked XR applications. They are widely expected to comprise the massively interactive live event (MILE) component of the so-called metaverse. 

The timing and shape of much of what may be in store for XR adoption over the long term remain open questions. However, the contours of this new trend in VR live streaming are no longer a matter of speculation. As noted by The Verge, VR live video has become “the gateway drug between what most people watch today and the immersive era of virtual reality that is just getting underway.” 

A historic turnaround

This may come as a surprise to anyone who has paid attention to past struggles in this space. For example, one of the more publicized setbacks involved NextVR, the technology vendor that for several years led the 360 VR viewing charge with regularly scheduled NBA game productions. Less frequent offerings involved Fox Sports, the International Champions Cup, the NFL and other entities.

NextVR hit a wall in 2018 when low viewership led to cancellation of an investment round and reported layoffs involving 40% of its staff. Apple acquired NextVR at a widely reported price of $100 million but has yet to divulge what it plans to do with the technology.

Other VR 360 disappointments included NBCUniversal’s use of Intel’s VR platform to provide VR coverage from the 2018 Winter Olympics. The result was widely panned by critics and garnered overall ratings of just 1.7 stars out of 5 in both Apple’s App Store and Google Play, according to Sports Business Daily. Making matters worse, considerable momentum was lost in 2020 as some sports leagues reduced or terminated their immersive viewing options amid the slowdown in live competition caused by the COVID-19 pandemic. 

But others, like the NBA, which switched to another supplier, Britain’s BT with its Premier League VR offerings, and NBCU’s Olympics productions, stayed the course. And now advances in production, transmission, rendering and other techniques have set the stage for a new level of immersive service performance. Such advances are supported by increasing broadband speeds, higher computer power and next generation codecs delivering greater compression efficiency. 

The commitments to moving ahead stem from pay-TV operators’ understanding that there actually is a real market for these services, provided the user experience (UX) meets consumer expectations. In 2018 Deloitte released the results of a global survey of sports fans. It showed that well over a third of the fans described as casual viewers, and more than half of those categorized as fanatics, said they would be more likely to watch a game if they had access to “VR front-row seats.” 

In a 2017 survey of U.S. consumers, IBB Consulting found that a third of respondents said they would watch live sports on VR all the time if they were given the option. In both surveys, these sentiments held even though the vast majority of respondents weren’t VR users. 

Since then, the VR user base has increased significantly. According to eMarketer, the number of active VR headset users in the U.S. jumped from 20.8 million in 2020 to 28.3 million in 2021 and will hit 32.7 million in 2023. Mordor Intelligence recently reported the global volume of VR device shipments went from 13.48 million in 2020 to 18.04 million in 2021. According to the same Mordor research, shipments are on a projected course to hit 112.62 million by 2026. 

Finally, Fortune Business Insights predicts the global spend on VR technology and content will increase at a 44.8% CAGR from $6.30 billion in 2021 to $84.09 billion in 2028. 

Breakthroughs enabling superior UX

From a UX perspective, it is clear why there’s renewed interest in VR 360. There are many more advances in the offing, including VR headsets with sunglass-like form factors and more realism in immersive experiences. 

Today services are only offered with three degrees of freedom (3DoF). This means users in any of multiple stationary virtual positions can watch and zoom in on anything in the field of play. Experts say 6DoF enabling viewers to walk around in the virtual space while taking everything in probably won’t kick in until later in this decade.

But, clearly, the threshold for providing gripping live 360 UX has been reached, and it has happened in very short order.

This was evident in the superior UX gains achieved by NBCU with its recent Olympics coverage of the pandemic-delayed Tokyo 2020 Summer Olympics in 2021 and the Beijing 2022 Winter Olympics. In Beijing, by all accounts NBCU’s VR by Xfinity app delivered a significantly improved 360 VR UX. The broadcaster covered over 150 hours of on-demand and live content supported by NBCU’s VR by Xfinity app.

Six cameras were used to capture live VR programming: one VR360 and five VR180 cameras. The Beijing Olympics programming covered the opening and closing ceremonies, figure skating, curling, freestyle skiing, snowboarding, short track skating, and ice hockey. All were delivered with what was described as unprecedented clarity and immersive engagement through the joint efforts of Comcast and Sky engineers and a vendor team including Intel, Tiledmedia, Cosm, Twitztel Design, Tebi, and others. 

A significant advance involved the use of multiple cameras offering 8K UHD resolution to capture the action at 7,680 x 3,840 pixels for the VR360 feed, and 7,680 x 4,320 for the VR180 signal. This resulted in about an eight-fold increase in pixel counts over comparable 4K camera feeds. This created a high-resolution VR video viewing experience that did away with the “screen-door” pixelation effect that occurs when content is rendered at 4K resolution on headsets in close proximity to viewers’ eyes. 

Tiledmedia camera rig

Source: The Broadcast Bridge

Use of 8K is made possible within current bandwidth limitations by the streaming technique known as “tiling,” an innovation that was used in the summer Olympics as well. Tiledmedia’s ClearVR platform, utilizing an evolved version of MPEG’s Omnidirectional Media Format (OMAF), breaks an equirectangular cube map encoding of the camera feeds. This is analogous to the flattening of spherical surfaces used in 2D global maps, into segments or tiles. 

These are combined to create an immersive, high quality experience using only the content needed to fill the user’s screen or viewport with each shift of the viewer’s gaze. Tiles are compressed and delivered at varying degrees of resolution in tandem with how the eye registers different parts of the immediate field of view (FOV) in real life. This has the advantage that only a portion of the stream is delivered at the highest resolution. 

As a result, the Olympics 8K VR transmissions operated at just 25 Mbps using High Efficiency Video Coding (HEVC), which is less than half the bitrate required for 8K transmissions to TV sets. The 8K streams could be decoded by any device capable of decoding 4K. 

Remarkably, the renderings of changes in the user’s FOV happen so fast against a low-resolution backdrop of the complete 1800 or 3600 viewscape that there’s barely any noticeable loss in quality, even at global distances. This is abetted by the fact that the aggregated VR camera feeds are replicated at local content delivery network (CDN) nodes worldwide.

BT, too, has adopted the 8K/tiling approach to delivering the expanding VR 360 portfolio in BT Sports. That approach will soon be incorporated with Eurosport as part of a 50-50 joint venture with the recently created Warner Bros. Discovery conglomerate. BT is also working with EE, its 5G mobile service unit, to develop new BT Sport VR apps on the mobile side.

Along with bringing VR 360 video streaming of soccer to 5G, the company is experimenting with AR overlays to conventional viewing of rugby and car racing. Furthermore, it has tested standalone holographic renderings of boxing matches for viewing on tabletops in the home.

What’s next

Clearly, these are exciting times for VR live streaming, both for service providers and viewers. However, successful brands and services will always be subjected to the threat of piracy in one form or another. Stay tuned for the second post of this two-part blog series, Protecting live-streamed XR services. We will review the VR content security requirements for live streaming of sports events to safeguard the services from the threat of piracy, and a solution that satisfies them all.