SDI VANC Caption Insertion

Carrying CEA-608 and CEA-708 captions through an SDI plant means embedding them as vertical ancillary (VANC) data, wrapped in a SMPTE ST 334-2 Caption Distribution Packet and mapped into the serial digital signal by SMPTE ST 334-1. There is no sidecar file at this stage: the caption bytes ride in the blanking of the video raster itself, one ancillary packet per frame, and a single wrong checksum or a header/footer sequence mismatch means a downstream decoder drops the service silently. This page is the SDI-domain endpoint of the Caption Muxing, Packaging & Delivery track: where the transport-stream and OTT pages package captions for files and CDNs, this one packages them for baseband. It covers the ANC packet layout, the internal structure of the CDP, the 10-bit VANC word encoding, and the Python that assembles a valid packet you can hand to an SDI card or a libklvanc-style serializer.

The problem is unforgiving because the format is positional and self-checked. Every byte of the CDP contributes to a mod-256 checksum, the ANC packet carries its own 9-bit checksum over its data words, and the caption service is only recovered if the header and footer sequence counters agree. Miss any one and the frame’s captions vanish without an error — the raster still clocks out, the picture is fine, and the caption disappearance surfaces only at an FCC Part 79 audit or a viewer complaint. The rest of this page turns each of those rules into code with a threshold you can assert against.

Problem Framing: What a VANC Caption Insert Must Guarantee

Insertion is correct only when four independent invariants all hold, and each one has a numeric or structural threshold. The CDP identifier must be exactly 0x9669 — a decoder that does not see those two bytes at the start of the user data words never enters the caption state machine. The cdp_length field must equal the actual byte count of the packet, or a strict parser truncates or over-reads the ccdata section. The CDP checksum is a mod-256 value chosen so the whole packet sums to zero; a nonzero residue is a discard condition. And the header and footer sequence counters must be identical for the frame — they are the CDP’s own tamper check that the packet was assembled and transported as a unit.

Layered above the CDP is the ANC packet’s own contract. The DID/SDID pair is 0x61/0x01, the registered identifier for CEA-708 caption data under SMPTE ST 334-1; swap the two and a receiver looking for 0x61 in the DID position finds 0x01 and skips the packet. The data count (DC) must equal the number of user data words, which for a caption insert is exactly the CDP length. The ANC checksum (CS) is a 9-bit sum over DID, SDID, DC and every UDW, taken mod 512. And every one of those values travels the SDI link as a 10-bit word: eight payload bits, an even-parity bit b8, and its inverse in b9. Get the parity wrong and the deserializer flags a word error before your checksum is ever consulted. These invariants are the format-specific analog of the reading-rate and drift thresholds enforced elsewhere in the SCC vs SRT vs WebVTT architecture tradeoff space — here the caption content is already fixed and the job is to package it byte-exactly.

Pipeline Stage & Prerequisites

VANC insertion sits at the very end of the caption path, downstream of format conversion and QC. By the time bytes reach this stage the caption service has already been decided — the cc_data triples exist — and the only remaining question is whether the wrapper is legal. That makes the stage a pure serialization problem, which is why it is expressed in struct and bytearray rather than a parsing library. The upstream cc_data itself typically originates from a CEA-608/708 elementary stream; the file-domain counterpart of that packaging is covered in embedding CEA-608 & CEA-708 in transport streams, and the same caption service is what an OTT path would instead hand to HLS & DASH caption delivery.

The per-frame mapping loop that feeds this serializer — turning a whole program’s cc_data stream into a sequence of ready-to-embed packets — is the subject of the companion page on mapping caption cues to VANC packets. This page builds and validates a single packet; that one runs the cadence across the timeline.

Step-by-Step Implementation

Step 1 — Assemble the Caption Distribution Packet

The CDP is built as one bytearray so that the length and checksum can be patched in place once the body is complete. The header carries the 0x9669 identifier, a frame-rate nibble, a flags byte, and the sequence counter; the ccdata section carries the per-frame triples; the footer repeats the sequence counter and terminates with the mod-256 checksum.

import struct

CDP_IDENTIFIER = 0x9669          # SMPTE ST 334-2 CDP identifier
CCDATA_ID = 0x72                 # ccdata_section id
CDP_FOOTER_ID = 0x74             # cdp_footer id

# SMPTE ST 334-2 cdp_frame_rate nibble codes
FRAME_RATE_CODES = {
    23.976: 0x1, 24.0: 0x2, 25.0: 0x3, 29.97: 0x4,
    30.0: 0x5, 50.0: 0x6, 59.94: 0x7, 60.0: 0x8,
}

def build_cdp(cc_triples, sequence, frame_rate=29.97):
    """Assemble one CDP. cc_triples: list of (cc_valid, cc_type, d1, d2)."""
    fr_code = FRAME_RATE_CODES[frame_rate]
    body = bytearray()

    # --- cdp_header ---
    body += struct.pack(">H", CDP_IDENTIFIER)            # 0x9669, big-endian
    body.append(0)                                       # cdp_length placeholder
    body.append((fr_code << 4) | 0x0F)                   # frame-rate nibble + reserved 1111
    body.append(0x40 | 0x02 | 0x01)                      # flags: ccdata_present, svc_active, reserved
    body += struct.pack(">H", sequence & 0xFFFF)         # cdp_hdr_sequence_counter

    # --- ccdata_section ---
    body.append(CCDATA_ID)                               # 0x72
    body.append(0xE0 | (len(cc_triples) & 0x1F))         # marker bits 111 + cc_count (max 31)
    for cc_valid, cc_type, d1, d2 in cc_triples:
        # 0xF8 marker | cc_valid<<2 | cc_type — CEA-708 cc_data triple
        body.append(0xF8 | ((cc_valid & 1) << 2) | (cc_type & 0x03))
        body.append(d1 & 0xFF)
        body.append(d2 & 0xFF)

    # --- cdp_footer ---
    body.append(CDP_FOOTER_ID)                           # 0x74
    body += struct.pack(">H", sequence & 0xFFFF)         # cdp_ftr_sequence_counter == header
    body.append(0)                                       # packet_checksum placeholder

    body[2] = len(body) & 0xFF                           # patch cdp_length = total bytes
    body[-1] = (256 - (sum(body[:-1]) % 256)) % 256      # checksum: whole packet ≡ 0 (mod 256)
    return bytes(body)

The two placeholders are the crux: cdp_length is written after the body is known, and the checksum is computed over every byte except itself so that the completed packet sums to zero. Because the header and footer both receive the same sequence argument, they are guaranteed to match — the failure mode this prevents is assembling a packet from two different frames’ worth of state.

Step 2 — Map bytes to 10-bit VANC words with parity

Every byte that leaves the SDI serializer is a 10-bit word. Bit 8 is even parity over bits 0–7, and bit 9 is the inverse of bit 8. That inverse guard is what lets a deserializer distinguish a real data word from the reserved 0x000/0x3FF flag values, which have no legal parity.

def parity9(value: int) -> int:
    """9-bit word: b0..b7 payload, b8 = even parity (SMPTE ST 291-1)."""
    v = value & 0xFF
    return ((bin(v).count("1") & 1) << 8) | v            # b8 makes total ones even

def to_10bit(value: int) -> int:
    """10-bit VANC word: adds b9 as the inverse of b8."""
    w9 = parity9(value)
    b9 = ((w9 >> 8) & 1) ^ 1                              # b9 = NOT b8
    return (b9 << 9) | w9

Step 3 — Wrap the CDP as an ANC packet

The finished CDP becomes the user data words of an ancillary packet. The ancillary data flag (0x000 0x3FF 0x3FF) is prepended by the physical serializer, but DID, SDID, the data count and the checksum are ours to compute. The ANC checksum sums the 9-bit parity-carrying values of DID, SDID, DC and every UDW, modulo 512.

DID_CEA708 = 0x61                # SMPTE ST 334-1 caption DID
SDID_CEA708 = 0x01               # CEA-708 SDID

def wrap_vanc(cdp: bytes) -> dict:
    """Wrap CDP bytes into a VANC ANC packet descriptor."""
    udw = list(cdp)                                       # user data words = CDP bytes
    dc = len(udw)                                         # data count = CDP length
    # ANC checksum: 9-bit sum of DID, SDID, DC and UDW (with parity), mod 512
    cs = sum(parity9(w) for w in [DID_CEA708, SDID_CEA708, dc] + udw) & 0x1FF
    return {
        "adf": (0x000, 0x3FF, 0x3FF),                    # ancillary data flag (added by serializer)
        "did": DID_CEA708,
        "sdid": SDID_CEA708,
        "dc": dc,
        "udw": udw,
        "cs": cs,
        # the line words the SDI card actually clocks out:
        "line_words": [to_10bit(w) for w in [DID_CEA708, SDID_CEA708, dc] + udw],
    }

The line_words list is the deliverable a DeckLink or comparable SDI SDK accepts: the data-carrying words already mapped to 10 bits. In practice you place this ANC packet on VANC line 9 for HD (1080i/720p) rasters and line 21 for SD — line 21 being the analog legacy line-21 position preserved into SDI.

Threshold Reference Table

Every fixed value the insertion asserts, with its source. Tune constants against this table, not against prose.

Verification & Test Pattern

A packet you cannot round-trip is a packet you cannot trust. The verification parses the CDP back out, re-derives the length and checksum, and asserts the header and footer sequence counters agree — the single most common silent corruption in a live plant. Pin it as a pytest so a refactor cannot weaken a field.

import struct
import pytest
from vanc_insert import build_cdp, wrap_vanc, DID_CEA708, SDID_CEA708

def parse_cdp(cdp: bytes) -> dict:
    assert struct.unpack(">H", cdp[0:2])[0] == 0x9669, "bad CDP identifier"
    assert cdp[2] == len(cdp), "cdp_length != actual byte count"
    assert sum(cdp) % 256 == 0, "CDP checksum residue is nonzero"
    hdr_seq = struct.unpack(">H", cdp[5:7])[0]
    ftr_seq = struct.unpack(">H", cdp[-3:-1])[0]          # footer: id, seq(2), checksum(1)
    assert hdr_seq == ftr_seq, f"sequence mismatch {hdr_seq} != {ftr_seq}"
    return {"length": cdp[2], "seq": hdr_seq}

TRIPLES = [(1, 0, 0x94, 0x20), (1, 0, 0x54, 0x45)]        # pop-on control + 'TE'

def test_cdp_round_trips():
    cdp = build_cdp(TRIPLES, sequence=42, frame_rate=29.97)
    meta = parse_cdp(cdp)
    assert meta["seq"] == 42

def test_anc_checksum_is_9_bit():
    cdp = build_cdp(TRIPLES, sequence=7)
    pkt = wrap_vanc(cdp)
    assert pkt["did"] == DID_CEA708 and pkt["sdid"] == SDID_CEA708
    assert pkt["dc"] == len(cdp)
    assert 0 <= pkt["cs"] <= 0x1FF                        # checksum fits 9 bits

def test_footer_tamper_is_caught():
    cdp = bytearray(build_cdp(TRIPLES, sequence=1))
    cdp[-3] ^= 0xFF                                       # corrupt footer sequence counter
    with pytest.raises(AssertionError):
        parse_cdp(bytes(cdp))

Troubleshooting / Failure Modes

Decoder ignores the service entirely : Root cause: the CDP identifier is not 0x9669 at the start of the UDW, usually because the ADF or a stray word shifted the offset. Fix: assert struct.unpack(">H", cdp[0:2])[0] == 0x9669 before wrapping, and confirm the serializer is not prepending its own header into the UDW region.

Captions flicker or drop every few seconds : Root cause: header and footer sequence counters disagree, so a strict receiver treats the CDP as corrupt and discards that frame. Fix: pass a single sequence value to both header and footer, as build_cdp does, and increment it exactly once per frame.

Wrong CEA-608 vs CEA-708 language decoded : Root cause: DID and SDID were swapped, so 0x01 lands in the DID position. Fix: keep DID = 0x61, SDID = 0x01; never derive one from the other. A receiver keys on DID first.

Intermittent word errors on the deserializer : Root cause: the 10-bit parity is inverted — b8 computed as odd parity, or b9 not set to the inverse of b8. Fix: use parity9/to_10bit; b8 must make the count of ones in b0–b8 even, and b9 must be its complement.

Checksum rejected downstream despite correct content : Root cause: mixing the two checksum domains — applying the CDP’s mod-256 rule to the ANC packet or vice versa. Fix: the CDP checksum makes the packet sum to 0 mod 256; the ANC CS is a 9-bit sum mod 512 over DID/SDID/DC/UDW. They are independent.

Wrong frame-rate code trips a cadence check : Root cause: the cdp_frame_rate nibble does not match the video raster (e.g. 0x5 for 30 on a 29.97 signal), so the receiver expects a different cc_count cadence. Fix: source the nibble from FRAME_RATE_CODES keyed on the real signal rate.

Operational Notes

At single-packet scale this is deterministic arithmetic, but a live insert runs at frame rate for the length of the program, and that turns two operational concerns to the front. First, the sequence counter is a rolling 16-bit value — it wraps every ~36 minutes at 29.97 fps — so treat wraparound as normal and never assert monotonicity across the boundary; only assert that a given frame’s header equals its footer. Second, empty frames still need a packet: a video frame with no new caption data must carry a padded CDP with valid-flag-clear filler triples so the ancillary cadence stays constant, which is exactly the padding logic the mapping caption cues to VANC packets loop applies frame by frame.

When the cc_data originates upstream from an SCC or elementary-stream source, decode it into clean triples first — the state-machine decode in parsing SCC with Python libraries produces the byte pairs this serializer expects — and validate the finished line words against a reference implementation such as libklvanc before trusting them on air. Keep the serializer stateless per frame: the only state that should cross a frame boundary is the sequence counter itself.

Related

• Mapping caption cues to VANC packets — the per-frame loop that emits a stream of these packets across a timeline.
• Embedding CEA-608 & CEA-708 in transport streams — the file-domain packaging of the same caption service.
• HLS & DASH caption delivery — where the OTT path carries captions instead of baseband VANC.
• SCC vs SRT vs WebVTT architecture — format tradeoffs that decide what content reaches this serializer.
• Parsing SCC with Python libraries — decoding upstream cc_data into the triples this page packages.

Part of: Caption Muxing, Packaging & Delivery — the caption packaging and delivery reference across baseband, transport-stream and OTT domains.