Supporting CEA-608 Closed Captions for Live Broadcasts using HLS Video

EdgePlayer and the CEA-608 Decoder

The EdgePlayer is a lightweight web video player solution that is designed to play HLS video through the HTML5 video element. On desktop browsers lacking HLS support, it uses a Flash polyfill built using an open source Flash plugin called Flashls. There were many challenges in creating a custom player but of particular interest was supporting in-band 608 captions for live broadcasts. CEA-608 captions is a standard for delivering closed caption data that has been used for analog television broadcast in North America for decades. Although it is sometimes considered to be outdated, mostly due to its character limitations and the inability for user customization, it is still in wide use for digital live and broadcast situations.

Problem

There were 3 areas that we needed to tackle to surface 608 captions to the user. The first was getting the caption data from the packets to the polyfill. Then we needed to decode the cc data into characters and commands. From there we could use the commands to position and correctly style the characters and render them to the user.

Implementation detail

At the time the player polyfill was being written, Flashls did not have any 608 caption support and the work being done to surface the cc data had no timeline for availability. However, FlasHLS contributor jlacivita had started to parse the cc data from frames and thus, we used this code as a starting point. Using a forked version of Flashls and jlacivita’s code, we were able to create a custom version of Flashls that delivered reliable 608 caption data to the polyfill. It is to be noted that this custom version of Flashls was used to create the decoder and renderer until Flashls released a version from their main fork that surfaced 608 caption data (v0.4.4.20)

Once we had caption data surfacing to the polyfill in the form of byte arrays, the task was to decode the bytes into renderable captions. Many 3rd party services provide flash based players with support for rendering in-band 608 captions often using a Adobe produced OSMF (Open Source Media Framework) based player. The OSMF player itself is open source and was a potential solution. However, it uses a caption decoder that is not open source and the added weight from using the OSMF player contradicted our goal of keeping the player and polyfill as lightweight as possible which made it less than ideal. Without the availability of a viable 3rd party solution, we created our own 608 decoder written in AS3 which is supplied to the polyfill as an external swc.

The polyfill processes the events from Flashls and passes the cc data to the decoder that is to be decoded. This data contains a set of two 8 bit bytes. These bytes contains either 2 standard characters, 1 special character, 1 extended character or a command. Once decoded, the characters and commands are surfaced from the decoder via event with the decoded data in the form of a JSON object that contains caption text, or command, row, position, text color, text decoration, and background color/transparency.

Rendering of the captions is done on the JS side which implements the 608 commands so the characters display in the correct position on the screen with the appropriate (inline) style, text decoration, and display mode (such as pop on or rollup). The benefit of rendering in JS is that it adds the ability to control position and apply custom styles to the captions. Generally, 608 caption styles are “burned in” and don’t comply with current FCC regulations that require captions be customizable per user preference. Rendering in JS provides the ability to fully customize the captions and it also mitigates issues such as collisions with UI elements.

You can check out the Decoder code on GitHub, which also contains the simplified JS renderer.


Shane Kerr | shane.kerr@redspace.com
Joe Park | joe.park@viacom.com

Joe J. Park

Read more posts by this author.