Gatherings Are On Hold… Get Streaming

As I write this, it is apparent that things are not going to get back to normal in the events industry any time soon.

The NSW Premier has stated that social distancing will be the “new normal” until a vaccine is out, and the Chief Medical Officer has stated that social distancing will be in place for at least six months, plus epidemiologists are openly stating that they cannot see a way for our international borders to be opened back up for at least twelve or more months.

This is because we might get the COVID-19 problem under control in Australia, but the rest of the world will take much longer, and at varying rates, so the risk is too high to allow foreigners into the country.

I really hope that I am wrong, but the event industries will remain dark for a very long time. This means that streaming and virtual meeting technologies have suddenly become widely used. For those in the AV industry, it presents an opportunity in an otherwise terrible market.

There are lots of way to do streaming, as well as lots of different destinations. I’m going to delve into some of the issues. I won’t get into the video and sound acquisition… you know how to do that.

The first step in the chain is encoding of the content. This can be done in software on a computer using external interfaces, made by companies such as Blackmagic Design, Aja, or Magewell.

Most modern computers are capable of streaming, but processor speed, threads, memory, and importantly, graphics processor memory and speed, will dictate how well they perform.

In terms of software, there are three well known options. The first is OBS Studio which is Open Broadcaster Software. It works on all platforms (Windows, MacOs and Linux), it is very functional, open source, and it is genuinely free!

It is relatively light on features, but OBS will comfortably handle any entry level streaming job. It does real-time source and device capture, basic scene composition, encoding, local recording, and of course, streaming.

For commercial products with more features, the main players are vMix and Wirecast.

vMix is a powerful application available for Windows only. It is a comprehensive software video mixer with a live streaming platform. It does have a learning curve, but in my view, vMix has emerged as the standard
in streaming software and it is made by Australians! We love that.

Currently up to version twenty-three, it is sophisticated and actively maintained; there isn’t much you cannot do on vMix.

Complex multi input productions can be put together, with picture in picture, titling, social media feeds, and integrated vMix Call that allows you to add remote guests to your production with high quality audio and up to HD video. The guests need nothing more than a laptop with Chrome installed.

vMix have more than two hundred tutorials on their Youtube channel and have a generous ninety day, full functioning trial at the moment whilst we are in isolation (normally sixty days… still very generous). vMix have an entry level version, right through to a sophisticated 4K version with numerous add-ons.

Wirecast is a Windows/Mac live video production program that can input and mix multiple audio and video inputs, add transitions, layers, titles, and other elements.

Currently on version thirteen, it can encode and output the streams to different services as well as record to disk. They too have a remote caller system, which they call Rendezvous, and social media feeds.

If you are on Mac, it is your only option other than OBS. The Wirecast trial is fully functional and not time limited. However, your broadcasts will contain both video and audio watermarks.

If you do your switching in a hardware external switcher, you might be better off with a dedicated hardware encoder.

Hardware encoders are built specifically for that purpose so they are likely to be more reliable than computers that rely on hardware, operating systems and software by others, and the bloat that brings. For example, Aja, Magewell and Kiloview make hardware encoders with SDI or HDMI inputs that are well priced and worth checking out.

A hybrid of these two approaches is a video mixer with a stream output.

Just last month Blackmagic Design released the Atem Mini Pro which is a new simple switcher complete with mutltiview that has its own encoder and Ethernet port. That means you can stream out of the switcher itself.

The gold standard for dedicated hardware combined with software
for streaming is Newtek’s Tricaster range. These are products that merge live video switching, broadcast graphics, virtual sets, special effects, audio mixing, recording, social media publishing and web streaming into an integrated appliance.

But what does an encoder do? It takes the video stream and converts that into a RTMP stream of usually H.264 video. RTMP stands for Real Time Message Protocol, which was developed by Macromedia (now part of Adobe).

It is quite an old TCP based technology that used to be the main method for streaming when combined with Flash. It is old and outdated but despite this, RTMP remains the industry standard protocol for transport of a stream to a Content Delivery Network (or CDN).

Every encoder can stream RTMP. A RTMP address is formatted like any other address but starts with rtmp:// instead of http://. For example, Facebook’s RTMP URL is rtmps://live-api-s.facebook.com:443/rtmp. The extra “s” on the rtmp is simply for a secure connection. As well as the address, you have a key as well, which is just like a password.

You can stream to free public CDNs like YouTube, Facebook, Instagram, Linkedin and Twitch, or paid services such as Vimeo Live, Wowza, or Dacast where you have many more user features, more control, and greater reliability.

All the CDNs have their benefits and disadvantages. For example, Facebook is fantastic for reaching new audiences, however you cannot let copyrighted music be on your stream because their bots will mute it and in some cases shut your stream down.

YouTube will do the same, quite often even if you have paid for the rights to use the music. Therefore you might be better off using a paid CDN provider like Wowza as their service comes with support as well as some other bells and whistles.

The CDN takes the RTMP feed and transcodes it normally into HTTP Live Streaming, known as HLS.

HLS is an adaptive bitrate streaming communications protocol developed by Apple. It is the most popular streaming format to end users because it is encoded at different bit rates, segmented into chunks, and sent to the client using an extended M3U playlist.

The adaptive playlist is actually a choice of resolutions and bitrates divided up into chunks. The viewer’s player chooses the resolution depending on the user’s internet bandwidth.

If they have a great internet connection, they’ll get the highest resolution chunks delivered and have a high quality image. However, if they have a poor internet connection, only low resolution, low bitrate chunks are delivered.

It doesn’t look as good, but at least it is there without stuttering. Also, HLS can traverse any firewall or proxy server that lets through standard HTTP traffic so it can be delivered reliably.

A CDN has another very important function. Each stream uses bandwidth to deliver. Say the stream from the encoder is 5 Mbps and you have 20 Mbps upload capacity. That means at best, you can deliver just four simultaneous outputs (if you are lucky!).

But what if you have a thousand viewers? You’d need a 5 gigabit upload internet connection.

A CDN gets around this bottleneck by transcoding the stream into small HLS chunks, making copies of the HLS stream, transferring them to delivery servers all over the world, and serving up the stream of HLS chunks to every single user watching the stream.

The viewer’s player plays these chunks out sequentially and for whatever reason, if a viewer’s internet connection gets worse, the CDN will simply start sending lower resolution chunks, thereby reducing the amount of data sent.

Equally, if the viewer’s bandwidth gets better, higher resolution HLS chunks are then sent.

CDNs do a lot of heavy lifting and are capable of massive scale. They do this through sheer scale and computing power. They take the incoming stream, transcode it, then they cache it on literally thousands of computers around
the world.

The reason they are all around the world is so that they are geographically close to the end users thereby ensuring a good experience for users worldwide. By the way, Netflix, Prime Video, Stan etc all use outsourced CDNs to deliver their content. Not to transcode, but to get the massive amounts of video data streams out to all their viewers.

I have built my own CDN using Amazon’s Web Services platform. I did this because I wanted a CDN that was focussed on the Australian region, plus I need a whole heap of features for my premium clients that I could not get cost effectively.

To start with, it has a primary and backup upload link so if one were to fail, it simply defaults to the backup. My connections are to the AWS edge server in Sydney, ensuring that the connection is quick.

The transcoding is done in the same room and farmed out to their other edge Cloudfront delivery servers around Australia and the world.

By building my own, I have ensured a good experience for my Australian users, which is my most important market, by making the round-trip path and delivery time as short as possible.

A lot of the other CDNs are focussed on the US markets; you cannot choose your upload edge location, and the rest of the world including Australia comes second.

The weakest link for streaming an event is nearly always the internet connection between your encoder and the CDN. Most internet connections are designed for big download, not big upload. Most users consume content rather than deliver it, so it makes sense to design the networks to accommodate big downloads, rather than uploads.

A typical 1080 25p RTMP stream uses between 5 Mbps to 10 Mbps of upload bandwidth. The widely accepted rule is that you want 100% headroom for uploading a stream so that means you want a minimum of 10 Mbps upload bandwidth available just for your 5 Mbps stream.

You absolutely want to test the upload speed of your venue’s internet connection.

We all know that the quality of internet connections can be affected by lots of things out of our control, and ISPs often engineer better access to speedtest.net to display better results than reality. I suggest you want even more headroom, especially in the peak times of the evening. (Check out my article on the NBN – link at the end of this article.)

A lot of people use speedtest.net but I prefer speedof.me. As mentioned previously, ISPs tune their networks so as to deliver better results to speedtest.net than the real world, indeed speedtest.net servers are often hosted by the ISPs themselves. That aside, I like speedof.me because it gives a better visual indication of your internet connection over time and you can clearly see the dips in speed, which is the speed you can rely upon.

Venue internet connections are sometimes great. But often they are terrible for lots of reasons. Firstly, they tend to be locked down for legitimate security reasons. Secondly, they’ll often have lots of users on a single internet connection which means the available bandwidth to you is variable. And lastly, venues simply won’t permit outsiders onto their LANs, especially if they are doing strange streaming things that they don’t understand. It is easier for them to say no.

In those cases, you go with 4G mobile broadband connections, the most unreliable connection of them all.

The phone companies use a mix of technologies and frequencies in their networks and the results are not always as you expect. What I am saying is that 4G is only a marketing term and there are different forms of it that perform differently.

To get around the variability of 4G connections, you can go with a bonded solution that redundantly shares the traffic across multiple internet connections. The LiveU Solo and Teradek VidiU Go are examples of these.

These devices have multiple connections by way of USB modems, Ethernet, and even Wi-Fi which take a video feed, encode the feed using their proprietary technologies, send it across the multiple connections to their own servers, which then re-assemble the feed and send it out as a RTMP feed to the CDN.

I have a LiveU Solo and it works great, but it is expensive to keep running because I maintain three 4G services, plus I pay an annual fee for access to LiveU’s bonding server. However, nothing comes close to its reliability and it is refreshing not having to rely on others for a reliable connection.

In my case, LiveU’s bonding server is in the same room as my CDN’s edge server in Sydney so once again, I have reduced the trip time as much as I possibly can.

You should never rely on Wi-Fi as your primary upload link. There are way too many variables that can impact the safe delivery of your real time stream. You need hardwired Ethernet.

The upload bandwidth required is a function of frame rate and resolution. That is, if you run with lower resolution and standard frame rates, you have a more reliable upload link because of less traffic.

I see a lot of people doing streams in 1080p at 60 frames per second. Why? Standard high definition television is often 720p at 25 frames per second so unless you are doing really high action sports, you definitely don’t need the high frame rates, and you probably need no more than 720p. At the time of writing, Facebook doesn’t deliver more than 720p anyway.

Lastly, there are three rules for successful streaming. Rule number one – Test. Rule number two – Test. And rule number three – Test. Leave yourself plenty of time to set up and test.

There are lots of gotchas with streaming that only become apparent from testing, so you need time to solve these. All CDNs have methods for you to test your stream thoroughly before going live to the public.

If you need technical support, there are quite a few user groups on Facebook. OBS, Wirecast and Vmix all have user groups, and there are people on there who have already had the problems you are working through.

Useful links:

OBS Studio. Free streaming software
www.obsproject.com

Wirecast. Streaming software
www.telestream.net/wirecast

vMix. Windows streaming software
www.vmix.com

URL for connection encoders
www.facebook.com/live/create

Youtube’s URL for connecting encoders
www.youtube.com/live_dashboard

Good speed testing site
speedof.me

CX articles I’ve written about streaming:
www.cxnetwork.com.au/mission-critical-show-networking
www.cxnetwork.com.au/networking-the-cables
www.cxnetwork.com.au/network-lan-scanning-see-whats-going-on
www.cxnetwork.com.au/the-nbn-not-to-blame-necessarily
www.cxnetwork.com.au/wi-fi-and-data-security-on-events

CX Magazine – May 2020   

LIGHTING  |  AUDIO  |  VIDEO  |  STAGING  |  INTEGRATION

Entertainment technology news and issues for Australia and New Zealand
– in print and free online www.cxnetwork.com.au

© VCS Creative Publishing