Caption Muxing, Packaging & Delivery

Once caption cues are parsed and validated, the hard problem shifts from what the text says to how it reaches a screen: the same verified cue model must be embedded as CEA-608/708 in an MPEG transport stream for cable and broadcast, serialized as IMSC1/TTML or segmented WebVTT for adaptive-bitrate players, and inserted as a SMPTE ST 334 caption distribution packet into SDI vertical ancillary data for playout to air. Each of those carriage paths has a different timebase, a different byte layout, and a different regulatory owner, so a cue set that passed every content check can still fail delivery because its transport wrapper is malformed. This guide is the engineering reference for that final stage — taking a frame-accurate cue model and getting it to air and to OTT players without breaking sync, positioning, or the audit trail.

The four encodings are not stylistic variants of one file — they are structurally incompatible carriers with different clocks. Transport-stream captions ride a 90 kHz PTS grid inside video SEI; SDI captions ride a line-locked VANC clock at the frame rate; HLS WebVTT segments carry an explicit X-TIMESTAMP-MAP bridging local cue time to that same 90 kHz stream clock; IMSC1 declares its own media time base in XML. The delivery stage’s job is to project one validated cue model onto whichever of these grids the distribution path demands, byte-exact, without letting the projection introduce the drift the parsing and validation stages worked to eliminate.

Regulatory & Engineering Context

Delivery format is not a free choice — it is legally constrained by the distribution path. For terrestrial and cable broadcast, captions must be carried as CEA-608 (the analogue “line 21” data service) and its digital successor CTA/CEA-708 DTVCC, because those are the formats decoders in deployed television hardware are built to render. CEA-608 fixes a 32-column by 15-row grid and rides in field 1 and field 2 of the ancillary data; CEA-708 wraps 608 compatibility bytes alongside native DTVCC service blocks. The physical transport of that data in the SDI domain is defined by SMPTE ST 334-1, which specifies the vertical ancillary (VANC) data mapping, and SMPTE ST 334-2, which defines the caption distribution packet (CDP) that actually contains the caption bytes. When the same signal moves over IP rather than coax, SMPTE ST 2038 carries that ancillary data — the CDP included — as a packetized elementary stream, so the VANC payload survives the transition to an IP production plant unchanged.

For IP-delivered and OTT video the governing formats are the web timed-text standards. W3C TTML1 is the XML timed-text vocabulary; the W3C IMSC1 profile constrains TTML into an interoperable subset that OTT devices are certified against; and W3C WebVTT is the segment-friendly format that HTTP Live Streaming carries natively. The two adaptive-bitrate transports impose their own container rules: HLS is specified in RFC 8216, which defines the EXT-X-MEDIA subtitle rendition and the X-TIMESTAMP-MAP tag; MPEG-DASH is specified in ISO/IEC 23009-1, which carries timed text as a dedicated text AdaptationSet.

The legal overlay ties the two worlds together. In the United States, FCC 47 CFR § 79.1 governs captioning of broadcast and cable programming, while § 79.4 extends parallel obligations to IP-delivered video programming that previously aired on television — meaning the same content is subject to the broadcast rules on its SDI path and the IP rules on its OTT path, simultaneously. A pipeline therefore cannot pick one delivery format and call the job done; it must produce the correct carriage for every path the asset travels, and prove each one is conformant. The upstream format tradeoffs that feed these decisions are laid out in SCC vs SRT vs WebVTT architecture, and the build-time enforcement that stops a non-conformant asset from ever reaching a mux is covered under CI/CD gating for caption builds.

Pipeline Stage Map

Muxing, packaging, and playout are the stages that run after the parse-normalize-validate chain has produced a trusted cue model. Everything here assumes that model is already frame-quantized and rule-checked; the delivery stage never re-edits caption content, it only re-encodes carriage.

The distinction between mux and package is the axis this guide turns on. Muxing interleaves caption data into a single multiplexed stream that also carries audio and video — CEA-608/708 bytes embedded in the video elementary stream of a transport stream, or a CDP inserted into the VANC space of an SDI signal. Packaging keeps caption data in separate, independently retrievable objects — a sidecar IMSC1 file, or WebVTT segments listed in their own media playlist — that a player pulls alongside the media. Broadcast paths mux; OTT paths overwhelmingly package. The four deep-dive sections below take one delivery encoding each.

Format & Target Overview

The table fixes the carriage properties that drive delivery code. These are the values the encoders below emit and the failure-mode checks assert against; keep them in one place rather than as magic numbers.

Delivery reference thresholds

Every fixed constant the delivery stage depends on, with its governing source:

Embedding CEA-608/708 in Transport Streams

The engineering problem is that a transport stream has no “caption track” — CEA-608/708 data lives inside the video, as cc_data triples wrapped in an A/53 user_data block that is itself carried in an H.264 SEI (or MPEG-2 picture user data). Each triple is a one-byte descriptor plus two caption bytes, and the descriptor’s cc_type field is what routes the two bytes to the right decoder path: types 0 and 1 are CEA-608 field 1 and field 2, types 2 and 3 are the start and continuation of CEA-708 DTVCC. Getting that field wrong sends 708 bytes to a 608 decoder and produces silent garbage. The mechanics — descriptor packing, parity, and the SEI wrapper — are worked end to end in embedding CEA-608 & CEA-708 in transport streams; the minimal encoder is below.

import struct
import subprocess

def odd_parity(byte: int) -> int:
    """CEA-608 payload bytes carry odd parity in bit 7."""
    b = byte & 0x7F
    ones = bin(b).count("1")
    return b | (0x00 if ones % 2 else 0x80)   # set bit7 to force odd count

def cc_data_triple(cc_type: int, b1: int, b2: int, cc_valid: int = 1) -> bytes:
    """One CTA-708 cc_data element.
    cc_type: 0=CEA-608 field1, 1=CEA-608 field2, 2=DTVCC start, 3=DTVCC data."""
    # first byte: 5 marker bits '11111' | cc_valid(1) | cc_type(2)  -- CTA-708 §4.4
    first = 0xF8 | (cc_valid << 2) | (cc_type & 0x03)
    return struct.pack("BBB", first, b1 & 0xFF, b2 & 0xFF)

def build_cc_field(pairs_608, service_blocks_708) -> bytes:
    """Assemble one frame's cc_data payload: 608 field-1 pairs then 708 blocks."""
    out = b""
    for hi, lo in pairs_608:
        out += cc_data_triple(0, odd_parity(hi), odd_parity(lo))  # 608 -> field 1
    for hi, lo in service_blocks_708:
        out += cc_data_triple(3, hi, lo)                          # 708 DTVCC data
    return out

def remux_with_a53(src: str, dst: str) -> None:
    """Remux a source that already carries A/53 captions into MPEG-TS,
    preserving CEA-608/708 as SEI user_data via libx264's a53cc pass-through."""
    subprocess.run([
        "ffmpeg", "-y", "-i", src,
        "-map", "0:v", "-map", "0:a",
        "-c:v", "libx264", "-a53cc", "1",   # ISO/IEC 13818-1 SEI user_data carriage
        "-c:a", "copy",
        "-f", "mpegts", dst,
    ], check=True)

The load-bearing detail is cc_type. A well-formed stream carries one 608 pair per frame per active field and interleaves the 708 service blocks in the same cc_data array, all tagged so a decoder reads only the types it understands. The -a53cc 1 flag tells libx264 to serialize captions it finds on the input frames into the standard A/53 SEI, which is what lets a downstream ATSC or cable decoder recover them; without it, a re-encode strips the caption side-data and the stream airs blank. Because the payload rides the video’s 90 kHz PTS clock, any PTS/PCR misalignment introduced at mux time drags the captions off the dialogue — a failure mode the last two sections return to.

IMSC1/TTML Packaging for OTT

The problem on the OTT side is the mirror image: instead of hiding caption bytes inside video, you serialize the cue model into a standalone XML document whose every timing and layout attribute must fall inside the IMSC1 profile, or certified players reject it. TTML is a large vocabulary; IMSC1 is the interoperable subset, and the two attributes that most often decide conformance are the parameter-namespace timeBase, which must be media so cue times are relative to the media clock rather than wall-clock or SMPTE timecode, and cellResolution, which IMSC1 pins at 32 15 to match the CEA-608 safe-area grid. Building the tree with lxml and explicit namespaces keeps those attributes in the right namespace — a common way to produce a document that validates as XML but fails as IMSC1. The conversion-from-WebVTT angle and the full profile-legality checks live in IMSC1 & TTML packaging for OTT.

from lxml import etree

TT  = "http://www.w3.org/ns/ttml"
TTP = "http://www.w3.org/ns/ttml#parameter"
TTS = "http://www.w3.org/ns/ttml#styling"
TTM = "http://www.w3.org/ns/ttml#metadata"
XML = "http://www.w3.org/XML/1998/namespace"

def build_imsc1(cues, lang: str = "en") -> bytes:
    """cues: iterable of (begin, end, text) with TTML clock-time strings.
    Emits an IMSC1-conformant timed-text document."""
    nsmap = {None: TT, "ttp": TTP, "tts": TTS, "ttm": TTM}
    tt = etree.Element(f"{{{TT}}}tt", nsmap=nsmap)
    tt.set(f"{{{XML}}}lang", lang)
    tt.set(f"{{{TTP}}}timeBase", "media")          # W3C TTML1 — media time base (required by IMSC1)
    tt.set(f"{{{TTP}}}cellResolution", "32 15")    # W3C IMSC1 — 32x15 cell grid, CEA-608 safe area

    head = etree.SubElement(tt, f"{{{TT}}}head")
    styling = etree.SubElement(head, f"{{{TT}}}styling")
    style = etree.SubElement(styling, f"{{{TT}}}style")
    style.set(f"{{{XML}}}id", "base")
    style.set(f"{{{TTS}}}color", "white")
    style.set(f"{{{TTS}}}backgroundColor", "black")
    style.set(f"{{{TTS}}}fontSize", "1c 1c")       # one cell — IMSC1 uses cell units

    body = etree.SubElement(tt, f"{{{TT}}}body")
    div = etree.SubElement(body, f"{{{TT}}}div")
    for begin, end, text in cues:
        p = etree.SubElement(div, f"{{{TT}}}p")
        p.set("begin", begin)                      # media-relative time, e.g. '00:00:04.200'
        p.set("end", end)
        p.set("style", "base")
        p.text = text
    return etree.tostring(tt, xml_declaration=True, encoding="UTF-8", pretty_print=True)

Serializing through lxml with a namespace map rather than string-concatenating XML guarantees the ttp: and tts: prefixes bind to the exact namespace URIs certified players match on; a document that puts timeBase in the default namespace is well-formed XML and non-conformant IMSC1. Declaring timeBase="media" is what makes the begin/end values line up with the packaged audio and video instead of drifting against a presentation clock — the same discipline the parsing stage applied to cue timing, now preserved through packaging.

HLS & DASH Caption Delivery

For adaptive-bitrate delivery the cue model becomes an independent rendition the manifest points at, and the recurring failure is time-origin, not text. HLS carries WebVTT as its own segmented media playlist referenced by an EXT-X-MEDIA:TYPE=SUBTITLES tag, and each WebVTT segment must begin with an X-TIMESTAMP-MAP header that maps the segment’s local 00:00:00.000 onto the presentation’s MPEG-2 stream clock at 90 kHz. If a stream’s first sample sits ten seconds into the 90 kHz timeline, the map value is 900000 (10 × 90 000) — get that offset wrong and every cue shifts by the error. DASH takes the packaging route instead, exposing timed text as a dedicated text AdaptationSet. Both encoders are below; the WebVTT segmentation details are expanded in HLS & DASH caption delivery.

from lxml import etree

def hls_timestamp_map(local_offset_s: float) -> str:
    """HLS RFC 8216 §3.5 — bridge WebVTT local time to the 90 kHz MPEG-2 clock.
    10.0 s of stream offset -> 900000 ticks."""
    mpegts = int(round(local_offset_s * 90000))     # 90 kHz caption clock
    return f"X-TIMESTAMP-MAP=MPEGTS:{mpegts},LOCAL:00:00:00.000"

def webvtt_segment(cues, local_offset_s: float) -> str:
    """One HLS WebVTT segment: header + timestamp map + cue blocks."""
    lines = ["WEBVTT", hls_timestamp_map(local_offset_s), ""]
    for start, end, text in cues:                   # start/end as 'HH:MM:SS.mmm'
        lines += [f"{start} --> {end}", text, ""]
    return "\n".join(lines)

def hls_subtitle_playlist(segments, target_dur: int = 6) -> str:
    """Media playlist for the subtitle rendition referenced by EXT-X-MEDIA."""
    out = ["#EXTM3U", "#EXT-X-VERSION:6",
           f"#EXT-X-TARGETDURATION:{target_dur}", "#EXT-X-MEDIA-SEQUENCE:0"]
    for name, dur in segments:                       # (uri, seconds)
        out += [f"#EXTINF:{dur:.3f},", name]
    out.append("#EXT-X-ENDLIST")
    return "\n".join(out)

def dash_text_adaptationset(lang: str, init: str, media_tmpl: str) -> bytes:
    """MPEG-DASH (ISO/IEC 23009-1) text AdaptationSet for segmented IMSC1/TTML."""
    ns = "urn:mpeg:dash:schema:mpd:2011"
    aset = etree.Element(f"{{{ns}}}AdaptationSet", nsmap={None: ns})
    aset.set("contentType", "text")
    aset.set("mimeType", "application/mp4")
    aset.set("lang", lang)
    rep = etree.SubElement(aset, f"{{{ns}}}Representation")
    rep.set("id", f"text-{lang}")
    rep.set("bandwidth", "2000")
    rep.set("codecs", "stpp")                         # IMSC1 in ISO-BMFF -> 'stpp'
    tmpl = etree.SubElement(rep, f"{{{ns}}}SegmentTemplate")
    tmpl.set("initialization", init)
    tmpl.set("media", media_tmpl)
    return etree.tostring(aset, pretty_print=True, encoding="UTF-8")

The X-TIMESTAMP-MAP is the single most error-prone value in HLS captioning: it is not a duration, it is an anchor tying local cue time to the same 90 kHz clock the muxed video uses, so it must be recomputed per segment against that segment’s true stream offset. On the DASH side the codecs="stpp" signal and contentType="text" are what let a player recognize the rendition as timed text rather than a broken media track; the IMSC1 payload inside each segment is exactly the document the previous section builds, now fragmented into ISO-BMFF.

SDI VANC Caption Insertion

Back on the broadcast side, playout to air means writing caption bytes into the vertical ancillary space of the SDI raster, and the problem is bit-level: the CEA-708 CDP has to be framed as an ANC packet with the right data identifier, then every byte expanded to a 10-bit ANC word carrying parity, then a nine-bit checksum appended — any one of which, wrong, makes the whole packet invalid and drops the captions off air. SMPTE ST 334-1 assigns the caption service DID 0x61 and SDID 0x01; SMPTE ST 291-1 defines the 10-bit word format, where bit 8 is even parity over bits 0–7 and bit 9 is its inverse. The full cue-to-VANC mapping is developed in SDI VANC caption insertion; the packet builder is below.

import struct

def build_cdp(cc_triples: bytes, seq: int) -> bytes:
    """SMPTE ST 334-2 caption distribution packet wrapping CTA-708 cc_data triples."""
    cc_count = len(cc_triples) // 3
    body = bytearray()
    body += struct.pack(">H", 0x9669)              # cdp_identifier -- SMPTE ST 334-2
    body.append(0x00)                              # cdp_length placeholder (filled below)
    body.append(0x4F)                              # frame_rate=0x4 (29.97) | reserved nibble 0xF
    body.append(0x43)                              # flags: ccdata_present | svc_active | reserved
    body += struct.pack(">H", seq & 0xFFFF)        # cdp_hdr_sequence_cntr
    body.append(0x72)                              # ccdata_section id
    body.append(0xE0 | (cc_count & 0x1F))          # marker '111' | cc_count
    body += cc_triples                             # the CTA-708 cc_data payload
    body.append(0x74)                              # cdp_footer id
    body += struct.pack(">H", seq & 0xFFFF)        # footer seq (must equal header seq)
    body[2] = len(body) + 1                         # cdp_length includes the checksum byte
    checksum = (256 - (sum(body) % 256)) % 256      # packet checksum: total % 256 == 0
    body.append(checksum)
    return bytes(body)

def anc_9bit(word: int) -> int:
    """Expand an 8-bit word to 9 bits with even parity in bit 8 -- SMPTE ST 291-1."""
    b = word & 0xFF
    b8 = bin(b).count("1") & 1                      # 1 when b0..b7 has odd count
    return (b8 << 8) | b

def build_vanc_packet(cdp: bytes, did: int = 0x61, sdid: int = 0x01) -> list[int]:
    """Frame a CDP as a 10-bit ANC packet: DID, SDID, DC, UDW..., checksum."""
    dc = len(cdp)                                   # data count = UDW length
    words9 = [anc_9bit(did), anc_9bit(sdid), anc_9bit(dc)]
    words9 += [anc_9bit(b) for b in cdp]            # user data words (the CDP bytes)
    checksum = sum(words9) & 0x1FF                  # 9-bit VANC checksum -- SMPTE ST 291-1
    words9.append(checksum)
    # promote every 9-bit word to a 10-bit ANC word: bit 9 = NOT bit 8
    return [(((w >> 8) ^ 1) << 9) | w for w in words9]

Three things have to be exactly right or the packet dies: the DID/SDID pair, which is how a downstream demux recognizes the ANC space as caption data at all; the parity in bit 8 of every word, because a receiver validates it and discards words that fail; and the checksum, computed over the nine-bit values of DID through the last user-data word. The cdp_length byte and the matching header/footer sequence counters are the CDP’s own integrity check — a mismatch there is one of the most common reasons a CDP is silently dropped by a hardware inserter even when the caption bytes themselves are perfect.

Failure Modes & Gotchas

These are the defects that pass content validation and still break delivery. Each has a cheap detection test and a known fix.

• 608/708 field mismatch. Tagging a caption triple with the wrong cc_type routes 708 bytes to a 608 decoder (or vice versa), producing blank or garbled output that looks like a decoder fault. Detect: assert cc_type ∈ {0,1} for 608 pairs and {2,3} for 708 blocks in every frame. Fix: separate the two services at assembly and tag each explicitly, as in build_cc_field.
• Missing caption_service_descriptor. A DTVCC stream without the PMT caption_service_descriptor leaves receivers unable to advertise or select the caption service, so menus show no captions even though the bytes are present. Detect: parse the PMT and assert the descriptor exists for each active service number. Fix: inject the descriptor at mux time with the correct service count and language tags.
• IMSC1 profile violations. A TTML document that uses features outside the IMSC1 text profile — unsupported tts: properties, wrong timeBase, absent cellResolution — is valid XML that certified players reject. Detect: validate against the IMSC1 profile, not just XML well-formedness. Fix: constrain output to the profile subset and pin timeBase="media" and cellResolution="32 15".
• X-TIMESTAMP-MAP offset wrong. Setting the MPEGTS anchor to a duration instead of the segment’s true 90 kHz stream offset shifts every cue; the canonical mistake is expecting 900000 (10 s) and emitting 0. Detect: cross-check the map value against the segment’s first video PTS. Fix: compute offset_s × 90000 per segment against real stream time, per hls_timestamp_map.
• VANC DID/SDID/checksum errors. A wrong DID/SDID pair, a bad parity bit, or a mis-summed nine-bit checksum makes an inserter discard the whole CDP, dropping captions off air with no error at the content layer. Detect: re-derive the checksum and parity on write-back and compare. Fix: build words through anc_9bit/build_vanc_packet and never hand-pack ANC.
• PTS/PCR misalignment. If the embedded caption SEI rides a PTS that has drifted from the program clock reference, captions lead or lag the dialogue even though every cue time is internally correct. Detect: measure caption PTS against audio PTS at known dialogue onsets. Fix: re-stamp on a single coherent clock at mux and keep the PCR within its jitter budget.
• Drop-frame vs non-drop at mux. Interpreting a 29.97 drop-frame source timecode as non-drop (or the reverse) at the mux injects roughly 3.6 seconds per hour of accumulating offset — inside tolerance early, a hard failure by the end of a program. Detect: compare computed end-of-program time against asset duration. Fix: normalize the timebase before mux, the same drop-frame arithmetic the parsing stage applies.

Compliance Telemetry & Audit Trail

Delivery only counts as done when each carriage path emits machine-readable evidence that it conformed, keyed to the clause it could have breached. Because a single asset now travels multiple paths — a transport stream to air, a WebVTT rendition and an IMSC1 sidecar to OTT — the telemetry record must name the target alongside the defect, so an audit can prove the broadcast copy and the IP copy were each conformant under their respective rules. The schema below maps every delivery defect to its governing clause and writes JSON for streaming consumers plus Parquet for the columnar store the automated QC validation & reporting layer aggregates.

import json, hashlib
import pyarrow as pa
import pyarrow.parquet as pq

DELIVERY_CLAUSES = {
    "cc_field_mismatch":     "CTA-708 §4.4 (cc_type routing)",
    "missing_svc_descriptor":"CTA-708 caption_service_descriptor",
    "imsc1_profile":         "W3C IMSC1 (text profile)",
    "timestamp_map_offset":  "HLS RFC 8216 §3.5 (X-TIMESTAMP-MAP)",
    "vanc_checksum":         "SMPTE ST 291-1 (ANC checksum)",
    "vanc_did_sdid":         "SMPTE ST 334-1 (DID 0x61 / SDID 0x01)",
    "pts_pcr_skew":          "ISO/IEC 13818-1 (PCR/PTS alignment)",
    "drop_frame_mismatch":   "SMPTE ST 12-1 (drop-frame timecode)",
}

def emit_delivery_telemetry(asset_id: str, target: str, defects: list[dict], out_prefix: str) -> str:
    """One record per delivery defect, each mapped to its clause, with a WORM hash."""
    records = [
        {
            "asset_id": asset_id,
            "target": target,               # 'mpegts' | 'imsc1' | 'hls' | 'dash' | 'vanc'
            "defect": d["code"],
            "clause": DELIVERY_CLAUSES.get(d["code"], "unmapped"),
            "measured": d["measured"],
            "threshold": d["threshold"],
            "severity": d["severity"],      # 'block' | 'warn'
        }
        for d in defects
    ]
    blob = json.dumps(records, sort_keys=True).encode()
    digest = hashlib.sha256(blob).hexdigest()          # chain-of-custody fingerprint
    with open(f"{out_prefix}.json", "wb") as fh:
        fh.write(blob)
    pq.write_table(pa.Table.from_pylist(records), f"{out_prefix}.parquet")
    return digest

Naming both the target and the clause on every record is what lets an FCC § 79.1/§ 79.4 inquiry be answered per distribution path rather than for the asset as an undifferentiated whole, and the content hash gives each report a fingerprint suitable for a write-once audit store. Because each record carries the measured value beside the threshold it failed, remediation tooling can block on severity == "block" defects while letting warnings through — the same fail-closed contract the CI/CD gating for caption builds stage enforces one step earlier in the pipeline.

Related

• CEA-608/708 in MPEG-TS — Embedding CEA-608 & CEA-708 in transport streams — cc_data triples and SEI user_data carriage.
• OTT timed-text packaging — IMSC1 & TTML packaging for OTT — profile-legal XML with media time base.
• Adaptive-bitrate delivery — HLS & DASH caption delivery — timestamp-map anchoring and text AdaptationSets.
• Playout to air — SDI VANC caption insertion — CDP framing and 10-bit ANC words.
• Upstream architecture — Broadcast captioning architecture & compliance — format and compliance decisions that set delivery targets.
• Trusted cue model — SRT, SCC & WebVTT parsing workflows — the parse and normalize stages the delivery stage consumes.
• Validation & reporting — Automated QC validation & reporting — where delivery telemetry is aggregated and archived.

Part of: Broadcast Media Closed Captioning & QC Automation — the mux, package and delivery stages of the end-to-end caption pipeline.