11 Jan 2021

Big Picture Deliver World’s First Live To Air xR Broadcast

Big Picture is always looking for new methods to allow creative and technical professionals to tell stories in fresh and innovative ways, especially during Covid-19. One area that certainly required a Covid-revamp was the award show format, and so with the Dally M Awards looming, thinking caps were put on by the guys at Fox Sports and Big Picture.

“Usually the Dally M Awards are staged in a large event venue, but with Covid-19 we had to rethink how we would stage the 2020 production,” remarked Paul Slater, Senior Producer at Fox Sports.

“We still wanted to have a big auditorium feel but we didn’t have the budget or resources to shoot something like that in reality.”

Paul turned to Big Picture for inspiration who in turn demonstrated how an xR broadcast could work. xR (extended reality) is a technology that allows you to blend virtual and physical worlds in live production environments to create fully immersive experiences.


This was a big step for Fox Sports, and a vote of confidence for Big Picture, as a live to air broadcast xR production had never been done anywhere in the world.

“We were approached by Fox Sports who had been looking at several technologies to integrate a virtual component into a live multi-camera environment,” explained Josh Moffat, Special Projects and Business Development at Big Picture.

“With only a short lead time until the event, we designed a technical solution around the disguise xR workflow.”

The event took place at Fox Sports in Sydney. The main studio setup comprised a 12m by 3m ROE Black Onyx 2.8mm LED Wall integrated into the disguise xR workflow.

The seamless extension of Big Picture’s real-world LED screens to the virtual world environments could only be done with disguise’s xR multi-camera registration workflow, allowing switching between camera perspectives and the LED content.

Three Panasonic 4000 camera chains ran a 1080P workflow which integrated downstream into Fox Sports 1080i transmission path. Big Picture’s UHD-2 PPU System with a ROSS Ultrix and Carbonite Ultra was at the heart of the system managing routing, monitoring, and conversion.

“Two Panasonic AW-HD150 UHD PTZ cameras were in a clean zone studio green screen to bring players into the main set virtually to interact with hosts,” added Josh.

“There were virtual objects in real-time in the 3D scene such as the leader board, player profiles and a virtual big screen fed from a mixture of disguise timeline and external EVS feeds.”

Five disguise gx 2c servers were used with Lightware HDMI 2.0 Matrix and Ross Ultrix SDI Matrix for failovers.

Stype RedSpy camera tracking devices were mounted on all three cameras, one studio pedestal, a jib and one Stedicam operated by Fox’s Glenn Steer, who did an amazing job with all the virtual graphics. The Stype RedSpy figures out where the camera is in 3D space and where it’s pointing – its position, angle, and field of view.

The Notch real-time engine then quickly drew the background scene from that point of view, before rendering the resulting image onto the screens on stage thus creating the illusion that the performers and props are actually in the virtual scene.

The Notch programming and integration was flawlessly executed by Ryan Sheppard, based in Canada. Big Picture had Ryan on comms in real-time for rehearsals, using a low-latency multiview stream and Clear-Com Agent-IC for communications.

He was then connected with remote desktop to the Notch file onsite, making changes as required. “We used Notch to render the 3D world designed by Fox Sports in real-time,” commented Ryan.

“Performance of the Notch scene was of utmost importance for a live to air multi-camera xR broadcast as there was no tolerance for dropped frames.

“We were able to achieve this level of performance by using a strategic combination of baked and unbaked textures that allowed for some dynamic elements while maintaining maximum performance of the scene.”

Working remotely on any project has its challenges. With Ryan located in Toronto, the biggest challenge for him was shifting his sleep schedule to match that of the Big Picture team in Sydney.

“However, once I was on the same sleep schedule, we were able to develop a workflow that allowed me to communicate via ClearCom Agent-IC as well as a VNC connection over AnyDesk to a computer on the network.

“That allowed me to network edit the block during programming and rehearsals if necessary. Once we had this workflow developed with Agent-IC, VNC, and low latency multiviews, I might as well have been sitting in the studio.”

Nathan Barnier, Senior Technician and Content Specialist at Big Picture, observed that there were several challenges with this project. Time was the major challenge, but a lack of a LED floor and the LED wall being flat and not curved had an impact on what is the usual xR workflow.

“Fortunately, we were able to build the cameras along with a test strip of the LED in the corner of the studio. This let us get ahead to achieve frame accuracy and seamless switching before the main LED wall build the night before going to air,” he revealed.

“Anyone with experience in the disguise xR workflow would acknowledge that frame accuracy and perfectly timed camera, tracking and, composite delay is extremely important, and the underlying factor to the successful execution of multi-camera xR.”

Having no LED floor is something that Ryan and Nathan tested a few weeks before the broadcast. Fox Sports wanted a gloss finish floor which would allow for reflections from the LED wall and the virtual scene to be realised in the environment.

A combination of alpha blend mode in the Notch block and a dummy LED floor in disguise was able to deliver a great outcome. Notch reflections of front plate objects such as players worked well on the high gloss acrylic chosen by Fox Sports.

Douglas Heriot ran the content from a MA Lighting grandMA2 console. The set up also ensured everyone was Covid-safe.

disguise xR is perfect for avoiding non-essential contact, mitigating the risks posed by traditional approaches to filming immersive visuals which would involve high-level, real-time in-camera shoots, green screen, and other VFX.

“Fox Sports were very happy with the result and I’m sure most people watching would have no idea of the technology used, as it worked so seamlessly,” said Paul.

“It was the perfect solution. We cover such a wide array of different sports and we like all of our hostings to look different to match the code, so to be able to change your set at the flick of a button is exciting to us and something we’ll be looking to use more in the future.”

Paul noted that Big Picture went above and beyond to rig and set up the system in a near impossibly short time frame.

“Their guys worked around the clock to make it happen which gave me all the confidence in the world that it was going to work!” he concluded.

Big Picture Crew

Project Manager/Technical Director: Josh Moffat

disguise xR Engineer: Nathan Barnier

Broadcast Engineers: Nick Bojdak, Olin Winton

disguise Operator/Programmer: Douglas Heriot

Notch Programming and Integration: Ryan Sheppard

LED: Cameron Richards, Justin Brown.

Big Picture

News from CX Magazine – December 2020   

Entertainment technology news and issues for Australia and New Zealand
– in print and free online

© VCS Creative Publishing


Published monthly since 1991, our famous AV industry magazine is free for download or pay for print. Subscribers also receive CX News, our free weekly email with the latest industry news and jobs.