Embedding CEA-608 & CEA-708 in Transport Streams

Embedding CEA-608 and CEA-708 captions in an MPEG transport stream means taking a decoded caption cue model and packing its bytes into the compressed video itself, so a television decoder finds and renders them frame-accurately at playout. A cue model that lives only as an SCC sidecar or a normalized Python object is invisible to a set-top box: the decoder never opens a separate subtitle file, it reads the caption bytes that were multiplexed into the picture user-data every frame. The engineering problem this page solves is that lowering — turning (start_frame, text, control_codes) tuples into the exact cc_data triples, DTVCC packets, and caption_service_descriptor signalling that ATSC A/53 requires, and getting them into the right elementary stream without corrupting the video. This is the in-band carriage stage of Caption Muxing, Packaging & Delivery, and it is where a correct upstream parse either survives to the viewer or is silently dropped by the mux.

The relationship that governs everything below is that CEA-708 does not replace CEA-608 on a DTV signal — it carries it. A single cc_data structure multiplexes both: legacy line-21 field bytes ride as cc_type 0 and 1, while native 708 DTVCC service blocks ride as cc_type 2 and 3, all in the same per-frame payload. Getting a program to render on both an old 608 decoder and a modern 708 one is therefore not two pipelines but one correctly interleaved byte stream.

Problem Framing: The Per-Frame Byte Budget

In-band captions are not a file appended to the stream — they are a fixed, tiny bandwidth reservation inside every coded picture. The ATSC cc_data() structure delivers 2 bytes per field per frame, and at NTSC’s 29.97 fps that is roughly 60 bytes per second per field. Every downstream constraint follows from that budget. A pop-on caption that a decoder must load into a back buffer, position with a Preamble Address Code, fill with text, and then flip to display costs a specific number of byte pairs, and if the author queues more control codes than the per-frame cc_count allows, the surplus is dropped and the caption renders half-formed.

The verdict at the receiver is binary and unforgiving: a caption either decodes cleanly into the on-screen grid or it is discarded. Three properties determine which. First, parity — every CEA-608 byte carries odd parity in bit 7, and a decoder treats a parity-failed pair as a null, so bytes constructed without parity vanish. Second, type discipline — 608 field-1 bytes must be tagged cc_type 0 and field-2 bytes cc_type 1, while 708 DTVCC bytes must be split correctly into a cc_type 2 packet-start pair followed by cc_type 3 continuation pairs; mislabel them and the decoder demultiplexes garbage. Third, signalling — even a perfectly interleaved payload is unreachable if the PMT does not advertise the service with a caption_service_descriptor, because the receiver’s caption menu is built from that descriptor. These are the same format tradeoffs weighed in SCC vs SRT vs WebVTT architecture, narrowed here to the question of physical carriage.

Pipeline Stage & Prerequisites

This stage runs after the caption artifact has been parsed and normalized but before final delivery packaging. Its input is a decoded cue model — ideally the canonical structure produced by parsing SCC with Python libraries — and its output is a transport stream whose video elementary stream carries the caption bytes and whose PMT signals them. The captions live inside the video ES, which is why a stream copy preserves them and a naive re-encode destroys them.

Install FFmpeg from a build with the libx264 and mpeg2video encoders enabled — both emit A53 caption side data when frames carry it, and a stripped build silently produces caption-free output. For the concrete FFmpeg recipe that muxes an SCC track into a container without re-encoding video, see injecting CEA-608 captions with FFmpeg; this page builds the byte structures that recipe carries.

Step-by-Step Implementation

Step 1 — Build cc_data byte pairs from CEA-608 control codes

Every CEA-608 byte pair — a control code such as Resume Caption Loading or a pair of text characters — becomes one cc_data triple: a header byte carrying cc_valid and cc_type, then the two payload bytes with odd parity applied. Pack the exact bit layout with struct so the marker bits land where a decoder expects them.

import struct

def odd_parity(byte: int) -> int:
    # CEA-608 line 21 bytes carry odd parity in bit 7; a parity-failed pair is dropped
    b = byte & 0x7F
    return b if bin(b).count("1") % 2 else b | 0x80

# CEA-608 control-code pairs on caption channel 1, field 1 (base bytes, pre-parity)
RCL = (0x14, 0x20)   # Resume Caption Loading — begin a pop-on caption in the back buffer
ENM = (0x14, 0x2E)   # Erase Non-displayed Memory
EDM = (0x14, 0x2C)   # Erase Displayed Memory
EOC = (0x14, 0x2F)   # End Of Caption — flip back buffer to display

def pack_cc_triple(cc_valid: int, cc_type: int, b1: int, b2: int) -> bytes:
    # ATSC A/53 Part 4 cc_data(): 5 marker bits (0b11111) then cc_valid(1) + cc_type(2)
    first = 0xF8 | ((cc_valid & 0x01) << 2) | (cc_type & 0x03)
    return struct.pack(">BBB", first, b1 & 0xFF, b2 & 0xFF)

def cc608_pair(pair, field: int = 0) -> bytes:
    # cc_type 0 = 608 field 1, cc_type 1 = 608 field 2; parity both payload bytes
    return pack_cc_triple(1, field, odd_parity(pair[0]), odd_parity(pair[1]))

Step 2 — Assemble a CEA-708 DTVCC packet

Native 708 text is not carried as bare byte pairs; it is framed as a caption channel packet containing one or more service blocks. The packet header holds a 2-bit sequence number and a 6-bit size code, each service block header holds a 3-bit service number and a 5-bit block size, and the whole packet may not exceed 128 bytes. Build the structure and enforce those ceilings explicitly.

def service_block(service_number: int, data: bytes) -> bytes:
    # CEA-708 service block header: service_number(3 bits) + block_size(5 bits)
    if not 1 <= service_number <= 6:
        raise ValueError("standard service numbers are 1-6; 7 requires an extended header")
    if len(data) > 31:                                   # block_size is 5 bits
        raise ValueError("a single service block holds at most 31 bytes")
    return bytes([(service_number << 5) | len(data)]) + data

def dtvcc_packet(sequence_number: int, blocks: bytes) -> bytes:
    # CEA-708 caption_channel_packet: sequence_number(2 bits) + packet_size_code(6 bits)
    total = len(blocks) + 1                              # + the 1-byte packet header
    if total > 128:                                      # DTVCC packet hard ceiling
        raise ValueError("a DTVCC caption channel packet may not exceed 128 bytes")
    # packet_size_code 0 => 128 bytes; otherwise packet_size = size_code * 2
    size_code = 0 if total == 128 else (total + 1) // 2
    header = ((sequence_number & 0x03) << 6) | (size_code & 0x3F)
    packet = bytes([header]) + blocks
    return packet + b"\x00" if len(packet) % 2 else packet   # pad to an even byte count

Step 3 — Interleave 708 into cc_data and wrap for the codec

The finished DTVCC packet rides in cc_data as a cc_type 2 packet-start pair followed by cc_type 3 continuation pairs, interleaved with the 608 compatibility pairs from Step 1. The per-frame triples are then wrapped in the codec-appropriate container: an ATSC A/53 cc_data() block (identifier GA94, type code 0x03) that becomes MPEG-2 user_data, or an ITU-T T.35 registered SEI (country 0xB5, provider 0x0031) for H.264/HEVC.

def dtvcc_to_cc_data(packet: bytes) -> bytes:
    # DTVCC bytes ride as cc_type 2 (packet start) then cc_type 3 (continuation)
    out = bytearray()
    for i in range(0, len(packet), 2):
        pair = packet[i:i + 2].ljust(2, b"\x00")
        out += pack_cc_triple(1, 2 if i == 0 else 3, pair[0], pair[1])
    return bytes(out)

def build_a53_cc_data(triples: bytes) -> bytes:
    # ATSC A/53 Part 4 cc_data(): reserved(1)+process_cc_data_flag(1)+add_data_flag(1)+cc_count(5)
    cc_count = len(triples) // 3
    if cc_count > 31:                                    # cc_count is 5 bits
        raise ValueError("cc_count exceeds 31 triples for a single coded picture")
    header = 0xC0 | cc_count                             # reserved=1, process_cc_data_flag=1
    return bytes([header, 0xFF]) + triples + b"\xFF"     # em_data byte + closing marker

def build_frame_payload() -> bytes:
    # One coded picture: 608 field-1 control pair + a 708 service block, both interleaved
    blk = service_block(1, b"HI")                        # service 1 = primary 708 caption
    triples = cc608_pair(RCL, field=0) + dtvcc_to_cc_data(dtvcc_packet(0, blk))
    return build_a53_cc_data(triples)

Step 4 — Mux into MPEG-TS with FFmpeg

Because in-band captions live inside the video elementary stream, a stream copy carries them intact into a transport stream, and re-encoding is only required when the source captions must be freshly injected as A53 side data. Drive FFmpeg from subprocess, keeping the timestamp grid the caption_service_descriptor will point at.

import subprocess

def remux_preserving_captions(src: str, dst: str) -> None:
    # Captions ride inside the video ES, so -c copy preserves 608/708 byte-for-byte.
    cmd = [
        "ffmpeg", "-y", "-i", src,
        "-map", "0",
        "-c", "copy",                 # no re-encode: the caption ES stays with the video ES
        "-copyts",                    # preserve the PTS grid captions are timed against
        "-f", "mpegts", dst,
    ]
    subprocess.run(cmd, check=True)   # non-zero exit raises CalledProcessError

The caption_service_descriptor (descriptor tag 0x86) that advertises the 608 and 708 services in the PMT is written by the muxer or the encoder’s caption metadata, not by the copy above; when it is missing, apply it at encode time so the receiver’s caption menu lists the service — the failure mode is covered below.

Threshold Reference Table

Every fixed value the byte structures above must honour, with its source. Tune constants against this table, not against prose.

The 0x61 DID/SDID row is the entry point to the serial-digital carriage path rather than the compressed one — mapping the same cue bytes into ancillary data is covered in SDI VANC caption insertion.

Verification & Test Pattern

The byte builders are pure functions, so pin them with assertions that a refactor cannot silently weaken: parity, the triple’s marker/type bits, and the two hard ceilings the standards impose.

import pytest
from embed import (odd_parity, pack_cc_triple, service_block,
                   dtvcc_packet, build_a53_cc_data, RCL, cc608_pair)

def test_odd_parity_sets_high_bit():
    assert odd_parity(0x14) == 0x94        # 0x14 has 2 one-bits -> parity bit set

def test_triple_marker_and_type_bits():
    triple = pack_cc_triple(1, 0, 0x94, 0x20)
    assert len(triple) == 3
    assert triple[0] == 0xFC                # 0xF8 | (cc_valid<<2) | cc_type 0
    assert triple[1:] == b"\x94\x20"

def test_dtvcc_packet_ceiling():
    with pytest.raises(ValueError):
        dtvcc_packet(0, bytes(200))         # > 128 bytes must be rejected

def test_service_block_size_limit():
    with pytest.raises(ValueError):
        service_block(1, bytes(40))         # block_size is 5 bits (max 31)

def test_cc_count_bounds():
    frame = build_a53_cc_data(cc608_pair(RCL))
    assert (frame[0] & 0x1F) == 1           # exactly one triple -> cc_count == 1

For an end-to-end check, decode the muxed stream with an independent implementation rather than the one that wrote it. ffprobe -show_streams reports the caption service on the video stream, and ccextractor input.ts -out=srt round-trips the embedded bytes back to text so you can diff them against the source cue model — an exact match proves the parity, typing, and interleave are correct.

def extract_and_verify(ts_path: str) -> str:
    # a53cc decode path: ccextractor demuxes 608/708 from the video ES back to SRT
    subprocess.run(["ccextractor", ts_path, "-out=srt", "-o", "decoded.srt"], check=True)
    with open("decoded.srt", encoding="utf-8") as fh:
        return fh.read()

Troubleshooting / Failure Modes

Captions vanish after a transcode : Root cause: re-encoding the video without preserving A53 side data drops the cc_data that lived in the source elementary stream. Fix: use a stream copy (-c copy) when only remuxing, and when re-encoding is unavoidable, use an encoder that carries caption side data and confirm with ccextractor on the output.

608 renders but 708 shows nothing (or vice-versa) : Root cause: the DTVCC bytes were tagged cc_type 0/1 (608) instead of 2/3, or the 608 compatibility pairs were omitted. Fix: interleave both — 608 field bytes as cc_type 0/1 and DTVCC as a cc_type 2 start pair plus cc_type 3 continuations — so both decoder generations find their stream.

No caption service appears in the TV’s menu : Root cause: the PMT lacks a caption_service_descriptor (tag 0x86), so the receiver never advertises the service even though the bytes are present. Fix: signal each service (language, 608 line-21 channel, 708 service number) in the descriptor at mux/encode time.

Every other character is dropped or garbled : Root cause: payload bytes were written without odd parity, so the decoder rejects the failed pairs as nulls. Fix: apply odd_parity to both bytes of every 608 pair before packing; 708 DTVCC bytes are 8-bit and must not be parity-masked.

Field 1 and field 2 captions overwrite each other : Root cause: both language services were emitted on cc_type 0 instead of splitting CC1/CC3 across field 1 and field 2. Fix: place the second 608 service on cc_type 1 (field 2) and keep the per-frame cc_count within budget.

Decoder stutters or drops packets under heavy caption load : Root cause: cc_count was pushed toward its 31-triple ceiling every frame, overflowing the decoder’s input buffer. Fix: spread control codes and text across successive frames so the sustained rate stays inside the 2-bytes-per-field-per-frame budget.

Operational Notes

The redundancy of carrying 608 inside a 708 signal is deliberate, not wasteful: it is the mechanism by which a single encode serves both a legacy line-21 decoder and a modern DTVCC one, and dropping the 608 compatibility bytes to save a few bytes per frame is the most common way to make a stream fail a decoder-targeting audit. When you must decide which services to prioritize under bandwidth pressure, weigh it against the compatibility matrix in CEA-708 vs CEA-608 decoder targeting.

Two carriage paths exist for the same cue bytes and they must stay consistent. The compressed-domain path on this page embeds cc_data in the video ES; the serial-digital path embeds the identical triples in VANC ancillary data under DID 0x61, and a facility that transcodes between SDI and compressed delivery has to preserve the mapping in both directions. Keep the caption authoring upstream of the split so both outputs derive from one cue model rather than re-deriving captions per format. Downstream of this stage, the OTT world abandons in-band carriage entirely for sidecar timed text — the tradeoffs there are the subject of IMSC1 & TTML packaging for OTT and segmented delivery in HLS & DASH caption delivery.

Related

• Injecting CEA-608 captions with FFmpeg — the concrete ffmpeg/ffprobe recipe that muxes an SCC track without re-encoding video.
• Parsing SCC with Python libraries — the CEA-608 decode that produces the cue model embedded here.
• SDI VANC caption insertion — carrying the same cc_data triples in serial-digital ancillary data (DID 0x61).
• IMSC1 & TTML packaging for OTT — the sidecar timed-text alternative to in-band carriage.
• HLS & DASH caption delivery — segmenting captions for adaptive streaming once they leave the transport stream.

Part of: Caption Muxing, Packaging & Delivery — the reference track for carrying, packaging and delivering caption data across broadcast and OTT.