Streaming Codecs for Video and Audio: What Broadcasters Need to Know in 2022

There’s a lot that goes on behind the scenes in live streaming. As a professional broadcaster, you should have at least a basic understanding of the technology that brings your broadcasts to life.

Even if you’re not a video engineer, the topic of streaming codecs is important. Codecs affect your streaming equipment needs, compatibility, budgets, bandwidth needs, and more. Staying on the cusp of next-generation codecs can be the difference between fading into obscurity or being on the cutting edge.

Today, we’ll cover everything you need to know about codecs. We will make sure you understand what is a codec is in a video streaming process.

We’ll discuss what a codec is, why they are important, and how they work before diving into several of the most common audio and video codecs. We will help you determine the best codec for streaming to fit your needs. We’ll also cover some associated topics, including wrappers and transport protocols.

Additionally, we’ll talk about the relationship between codecs and bandwidth. To wrap it up, we’ll share some of our recommended codec settings for online video and live streaming.

Are you ready to jump into the world of streaming codecs?

Please keep in mind that this post covers a topic that is subject to regular new developments. This post reflects the most accurate information as of August 2022.

Table of Contents:

• Streaming Codecs: The Basics
• Common Audio and Video Codecs
• Wrappers
• Transport Protocol
• Codecs and Bandwidth
• Next-Generation Streaming Codecs
• Recommended Codec and Settings for Live Streaming
• Conclusion

Streaming Codecs: The Basics

When you record a video, RAW digital video files are generated. RAW digital video files are composed of a series of still pictures. When these videos are played in succession at a high speed, you get a video.

Similarly, RAW digital audio is simply a collection of measurements of the magnitude of various tones. When played together, you get understandable audio. 

This RAW data is incredibly rich in information. However, that also means it’s bulky and unwieldy. A video composed of thirty images per second quickly becomes extremely large. Professional-grade cameras that capture RAW video can easily create terabytes of data from a single day of shooting. The same is true of RAW audio recorders.

While this is valuable in certain industries, it’s a liability for online video and live streaming. Viewers’ internet connections, mobile devices, and home computers simply aren’t powerful and fast enough to process this data. Storage and bandwidth are expensive, and RAW files tear through it.

Codecs offer a solution to this file size and speed issue. The term codec stands for “coding/decoding.” Essentially, a codec is a mathematical process that reduces digital file sizes by “throwing out” or reducing unnecessary data. The complex part is doing this intelligently while retaining as much quality as possible. That is what codec video is. 

Different codecs have different ways of doing this. One basic method that codecs use is to compare each frame to the previous frame. If data remains the same (for example, there’s a dark corner of the frame in both images), it can be tossed from the second frame. This is a simplified example, but it helps you understand how a codec works.

Over time, the methods have improved. However, codecs that compress video to smaller sizes while retaining more quality tend to require more processing power—both to encode and decode.

Common Audio and Video Codecs

As we mentioned, there are many different codecs available. There are a few main codec types that are commonly used for online audio and video. 

Some common audio and video codecs include:

H.264/AVC

The most used codec and encoding output used today are H.264 files. It is also referred to as Advanced Video Coding (AVC). This we developed by Telecommunications Union and the International Organization for Standardization/International Electrotechnical Commission (ISO/IEC) Moving Picture Experts Group. 

H.264 has become so popular because it isn’t just used for streaming; it is used for Blu-ray disks and cable broadcasting. This is more than likely due to who developed it. 

Plus, H.264 often works together with AAC audio codec, and can work with .mp4, .mov, .F4v, .3GP, and .ts containers. 

The thing that makes H.264 so popular is that it can be used on virtually any device – any browser, any device – can play H.264 videos. It is widely supported, and publishers know how to use it. 

According to Bitmovin’s 2022 Video Developer report, over 91% of those who took the survey use H.264. 

The downside to this is that it doesn’t work well with 4K video or with HDR content. It works well with low-latency streaming. It is commonly used with HTTP-based and WebRTC-based applications. This format has been around the longest and looks to stay that way for a while, as it can work with virtually any device. 

V9P

Google developed the V9P as an open-source, royalty-free alternative to H.264. It was released in 2013. 

Since being released, support for VP9 has increased. It is supported on: 

• YouTube
• Chrome browser
• Android phone
• Mozilla’s Firefox browser
• Apple’s Safari browser
• All new iOS devices
• Samsung, Sony, LG, Roku TVs

In fact, more than 90% of Chrome-encoded WebRTC video uses either VP9 or VP8, its predecessor. It ranks 2nd behind H.264/AVC codec regarding accessibility on devices. This is largely driven by Google using it for YouTube and Netflix’s adoption of this codec. 

It is a better version of AV1, which we will review below, as more devices support this codec. It is a higher-quality compression technology than H.264/AVC and is more compatible with devices than most other H.264 alternatives. 

Plus, it works really well with 4K streaming. 

H.265/HEVC

The SO/IEV Moving Picture Experts Group made H.265 a successor to H.264. It is also called High-Efficiency Video Coding. 

The big difference from H.264 is that it is made to create smaller files, uses less bandwidth, and supports high-resolution streaming. Although this is designed as a successor to H.264, it is only used by about 10% of encoded files. 

There has been some confusion about its royalties and what developers will have to pay for using the codec, which has stopped its widespread adaption. Royalty confusion actually helped spur the development of the AV1 codec.  

It is a good codec to use if you want to deliver 4K or HDR content. 

AV1

Major tech players are Amazon, Netflix, Cisco, Microsoft, Google, and Mozilla came together to form the Alliance for Open Media. They then created AV1, which is a royalty-free and open-source alternative. 

It is designed to be more efficient than H.264 and H.265. It works well with high-quality content but has long encoding times, making it more expensive to use.

H.266/ VVC

H.266, also referred to as Versatile Video Coding, or VVC, was just created in 2020. It is made to use instead of H.264 and H.265; however, royalty issues impede its widespread adoption. Confusion around royalties has stopped its widespread adoption. 

It is made to provide bandwidth savings while delivering high-quality content. 

ACC

ACC is regarded as the best audio codec that you can use. It was made in the late 1990s. Even though it was made decades ago, it still creates great audio quality using low bitrates. 

It is also widely compatible with all devices, which is why it continues to be the most used audio codec. 

MP3

The original codec that helped to drive media sharing on the internet started with MP3. MP3 isn’t used as a codec anymore; most people identify it as a file type. ACC is so much more effective.

H.264 Profiles

Since H.264 is the most popular video codec for online video, it’s worth mentioning a key element of it: profiles. Profiles are different versions of the H.264 codec. There are many different H.264 profiles, but the most common are Baseline, Main, and High.

Baseline and Main were originally created for low-powered devices. However, today the High profile is an excellent choice for most live video broadcasts. High-profile H.264 is the best codec for streaming video live. 

Wrappers

Let’s clear up one common confusion. The file extension at the end of a digital file often doesn’t tell you anything about the codec. Common filenames for media end in .mp4, .mov or .avi.

These are called “wrappers” or “container formats.” A wrapper allows a file to contain multiple different elements, such as subtitles and metadata. The container format you use isn’t particularly important for online videos. 

If in doubt, use .mp4. It’s the most widely supported format in wide use today. mp4 is the best video format for web streaming, as it will work with most other tools necessary for video streaming. 

Transport Protocol

As you can tell, internet video requires a series of technologies built on top of each other to solve new challenges. The transport protocol falls into this category.

Transport protocols are methods for streaming multimedia over the internet. They must be understood by both the sender and receiver in order to function.

There are two very common protocols in use today in the live stream industry. The first is the HLS streaming protocol, which is what we use at Dacast to deliver streams to viewers. HLS is widely compatible and provides good performance. 

The second protocol is RTMP (real-time messaging). This is an aging standard, but it’s still widely used for stream ingestion.

Protocols are regularly updated as technology advances, and MPEG-DASH is one that is on the rise. 

Codecs and Bandwidth

As you’ve learned, codecs help compress files to make them easier to transmit over the internet. Internet speeds are increasing, but video files are getting bigger as well. 4K video is becoming more common, as are HDR and HFR. Each of these advances in video technology requires more space. 

The size and quality of your video stream determine how much bandwidth you use. Higher bitrates (which correspond to higher quality) require more bandwidth to send. 

You’ll need to ensure your internet connection is fast enough to stream at your desired bitrate. You may also want to ensure your viewers have a quality viewing experience by streaming in multiple bitrates at once. Multiple bitrates allow the quality of the video to adjust to fit your viewer’s internet speeds and device abilities. 

Next-Generation Streaming Codecs

Your bandwidth usage is dependent on the codec you use. There is a variety of next-generation video and audio codecs in development today. 

Let’s mention a few briefly. 

1. HEVC, or H.265:

H.265 is the successor to H.264. It promises to deliver video with about double the quality/efficiency ratio of H.264. It is the future of streaming codec.

2. AV1:

Another forthcoming codec to check out is AV1. This open-source alternative to H.265 provides better quality and maybe ubiquitous in a few more years. It is a strong contender to become a top video codec in years to come. 

3. Opus

Opus is an up-and-coming audio codec. The Opus codec delivers better audio quality at every bitrate compared to other audio codecs. Opus codec is free and open-source and natively supported on iOS and Android.

Recommended Codec and Settings for Live Streaming

As we mentioned, codecs are an ever-changing technology, so the “recommended” codecs change relatively quickly.

For now, we recommend that most broadcasters use the H.264 codec for video and the AAC codec for audio. These are the best streaming codec for both live and on-demand video content delivery.

These recommendations will likely change in the future as codec technology develops, but as of right now, these are the best choices to reach the maximum possible audience with your OTT video.

For other encoder settings such as bitrate, resolution, sample rate, and so on, check out our blog post that covers recommended encoding settings.

Conclusion

Now that you’re up to date on everything you need to know about streaming codec, you’re one step closer to your first professional broadcast.

In this article, we’ve covered how codecs work, related technology, and the ideal codecs for use in online broadcasting today. We hope that the information we’ve shared gives you a bit more confidence in the more technical aspects of professional broadcasting and a better understanding of codec for video streaming. 

If you are just getting into professional broadcasting, you’ll need a live streaming platform like Dacast to get started. 

We invite you to give Dacast a try risk-free for 14 days. All you have to do is sign up, and you can start streaming in minutes. No credit card is required. Just click the link below to start streaming live in a matter of minutes!

Get started for free   

Any questions, comments, or ideas? Let us know in the comments section, below! And for regular tips on live streaming and exclusive offers, feel free to join our LinkedIn group.

Thanks for reading, and as always, best of luck with your live streams!