HLS & DASH Caption Delivery

Adaptive-bitrate players reject a caption track that is not segmented and time-mapped to their own media clock: the cues either fail to load, load onto the wrong track, or drift steadily against the video because the WebVTT local timeline never got anchored to the 90 kHz MPEG-2 presentation clock. This page closes that gap for both major streaming stacks — Apple HLS and MPEG-DASH — by turning one monolithic WebVTT (or IMSC1) file into a segmented, manifest-referenced subtitle rendition that a player can select, fetch per segment, and render frame-accurately. The two protocols disagree on almost every mechanical detail: HLS carries subtitles as a #EXT-X-MEDIA:TYPE=SUBTITLES rendition of sidecar .vtt segments each stamped with an X-TIMESTAMP-MAP, while DASH declares a text AdaptationSet in the MPD and slices IMSC1 or WebVTT into fragments described by a SegmentTemplate. Getting either wrong produces the same symptom in QA — captions that are present but misaligned — so this is the delivery-side counterpart to the packaging work that precedes it in Caption Muxing, Packaging & Delivery.

Two delivery models exist and they are not interchangeable. In-band captions (CEA-608/708) ride inside the video elementary stream and are decoded by the player’s built-in 608/708 decoder — no separate track, no manifest entry, covered under embedding CEA-608 & CEA-708 in transport streams. Sidecar captions (WebVTT for HLS, IMSC1/TTML or WebVTT for DASH) are independent, separately-fetched renditions selected from the manifest, which is what this page builds. Most OTT delivery today uses sidecar text because it decouples the caption track from the video ladder, supports mid-stream language switching without re-encoding, and lets a single caption file serve every bitrate variant.

Problem Framing: What Adaptive Delivery Demands of a Caption Track

A progressive-download caption file has one job: match the media’s zero. Adaptive delivery imposes four additional constraints, each with a numeric contract.

Segment alignment. Every rendition in an adaptive presentation — video, audio, subtitles — is sliced on a shared segment boundary so the player can switch variants at any segment edge. If video segments are 6 seconds, the subtitle rendition must also be 6-second segments with identical boundaries; a mismatch of even one segment shifts caption fetches out of step with the player’s buffer model. The typical target segment duration is 6 seconds (#EXT-X-TARGETDURATION:6), matching the Apple HLS authoring guidance.

The 90 kHz clock anchor. WebVTT cue timestamps are local to the file and start conceptually at zero. MPEG-2 systems — the timeline HLS media segments live on — count in a 90 000 Hz clock, so 10 seconds of media time is exactly 900000 ticks. Every .vtt segment must carry an X-TIMESTAMP-MAP=MPEGTS:<pts>,LOCAL:00:00:00.000 header that pins the file’s local zero to a point on that 90 kHz timeline. Apple’s convention places the first sample at PTS 900000 (a 10-second startup offset), so the standard anchor is MPEGTS:900000,LOCAL:00:00:00.000. Omit it and hls.js and native Safari fall back to assuming local zero equals media zero, so any presentation offset becomes visible caption drift.

Manifest-declared selection. The player must be able to discover, name, and default-select the caption track from the manifest without downloading a single cue. HLS does this with #EXT-X-MEDIA attributes (GROUP-ID, NAME, LANGUAGE, DEFAULT, AUTOSELECT, FORCED); DASH does it with AdaptationSet attributes and child Role descriptors. A GROUP-ID that no #EXT-X-STREAM-INF references, or a DASH AdaptationSet with no Role, produces a track that exists but is never offered.

Language labelling. Both protocols expect BCP-47 language tags (en, es-419, fr-CA) — not the three-letter ISO 639-2 codes that broadcast SCC workflows often carry. The tag drives the player’s language menu and any accessibility auto-selection, and a tag the player cannot resolve silently demotes the track to an unnamed entry the viewer will not recognize.

These four constraints are why a caption file that plays perfectly on a progressive download can be invisible or misaligned the moment it is dropped into an adaptive presentation. None of them touch the cue text itself — they are entirely about how the track is sliced, clocked, named, and referenced. That separation is deliberate: it means the same authored, QC-passed cue list can be delivered to both HLS and DASH by two mechanically different back ends that never re-open the words.

The reading-rate, line-length and sync-tolerance rules from upstream QC still apply to the cues themselves; this page assumes the WebVTT it segments has already passed WebVTT cue extraction & validation, so the work here is purely about slicing and time-mapping a known-good cue list.

Pipeline Stage & Prerequisites

Segmentation and manifest generation run after caption authoring and QC but before origin publish. At this stage the cue list is final and frame-quantized; the output is a directory of segments plus a manifest, ready to sync to a CDN origin. Run the segmenter in parallel with the video/audio packager, but pin the same segment duration and start offset across all three so boundaries align.

Install from wheels and pin exact versions; a webvtt-py minor bump can change how it normalizes cue whitespace, which changes segment byte output and breaks any golden-file comparison downstream.

Step-by-Step Implementation

Step 1 — Segment the WebVTT and stamp each segment’s X-TIMESTAMP-MAP

Read the source cues, bucket them into fixed-duration segments, and write each .vtt fragment with a WEBVTT header and the 90 kHz time-map. A cue that straddles a boundary is written into every segment it overlaps so it never disappears mid-display.

import webvtt

CLOCK_HZ = 90_000          # MPEG-2 system clock — 90 kHz; 10 s == 900000 ticks
PTS_OFFSET = 900_000       # Apple HLS convention: first sample at PTS 900000 (10 s startup offset)
SEG_SECONDS = 6.0          # match the video ladder's #EXT-X-TARGETDURATION

def ts_to_seconds(ts: str) -> float:
    # W3C WebVTT timestamp grammar: HH:MM:SS.mmm
    hh, mm, rest = ts.split(":")
    ss, ms = rest.split(".")
    return int(hh) * 3600 + int(mm) * 60 + int(ss) + int(ms) / 1000

def timestamp_map(pts_ticks: int) -> str:
    # HLS spec: pin WebVTT LOCAL zero to a point on the 90 kHz MPEG-TS timeline
    return f"X-TIMESTAMP-MAP=MPEGTS:{pts_ticks},LOCAL:00:00:00.000"

def segment_webvtt(src_path: str, out_dir: str, seg_seconds: float = SEG_SECONDS):
    captions = list(webvtt.read(src_path))
    total = max(ts_to_seconds(c.end) for c in captions)
    n_segments = int(total // seg_seconds) + 1
    segments = []
    for idx in range(n_segments):
        seg_start = idx * seg_seconds
        seg_end = seg_start + seg_seconds
        # A cue belongs to this segment if it overlaps [seg_start, seg_end) at all,
        # so boundary-spanning cues are duplicated into both segments (never dropped).
        members = [c for c in captions
                   if ts_to_seconds(c.start) < seg_end
                   and ts_to_seconds(c.end) > seg_start]
        # Constant map: LOCAL zero maps to program zero, which sits at PTS_OFFSET.
        header = ["WEBVTT", timestamp_map(PTS_OFFSET), ""]
        body = [f"{c.start} --> {c.end}\n{c.text}\n" for c in members]
        name = f"seg_{idx:05d}.vtt"
        with open(f"{out_dir}/{name}", "w", encoding="utf-8") as fh:
            fh.write("\n".join(header) + "\n" + "\n".join(body))
        segments.append((name, seg_seconds))
    return segments

The X-TIMESTAMP-MAP value here is constant across segments because the cues keep their absolute timestamps: local zero always maps to the same PTS anchor. The alternative — rebasing each segment’s cues to segment-local time and computing a per-segment MPEGTS offset — is the deep-dive in WebVTT segmentation for HLS, which also handles 33-bit PTS rollover and non-zero stream start offsets.

Step 2 — Generate the subtitle rendition playlist and the master #EXT-X-MEDIA line

The subtitle rendition is its own media playlist listing the .vtt segments; the master playlist references it with a #EXT-X-MEDIA:TYPE=SUBTITLES entry whose GROUP-ID must also appear on every #EXT-X-STREAM-INF variant that offers it.

import math

def write_media_playlist(segments, out_path: str, seg_seconds: float = SEG_SECONDS):
    target = math.ceil(max(d for _, d in segments))   # #EXT-X-TARGETDURATION >= max #EXTINF
    lines = ["#EXTM3U", "#EXT-X-VERSION:6",
             f"#EXT-X-TARGETDURATION:{target}",
             "#EXT-X-MEDIA-SEQUENCE:0", "#EXT-X-PLAYLIST-TYPE:VOD"]
    for name, dur in segments:
        lines.append(f"#EXTINF:{dur:.3f},")
        lines.append(name)
    lines.append("#EXT-X-ENDLIST")
    with open(out_path, "w", encoding="utf-8") as fh:
        fh.write("\n".join(lines) + "\n")

def ext_x_media(group_id: str, name: str, language: str,
                uri: str, default: bool = False,
                autoselect: bool = True, forced: bool = False) -> str:
    # HLS: TYPE=SUBTITLES rendition. BCP-47 LANGUAGE; DEFAULT/AUTOSELECT drive selection.
    yn = lambda b: "YES" if b else "NO"
    return (f'#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="{group_id}",'
            f'NAME="{name}",LANGUAGE="{language}",'
            f'DEFAULT={yn(default)},AUTOSELECT={yn(autoselect)},'
            f'FORCED={yn(forced)},URI="{uri}"')

# The variant streams must name the SUBTITLES group so players offer the track:
STREAM_INF = ('#EXT-X-STREAM-INF:BANDWIDTH=3200000,CODECS="avc1.640028,mp4a.40.2",'
              'SUBTITLES="subs"')

AUTOSELECT=YES lets the player pick this track when the OS caption language matches; DEFAULT=YES forces it on at start. Set at most one rendition per group to DEFAULT=YES. FORCED=YES is reserved for narrative subtitles (foreign-dialogue burn-ins) that must show regardless of the caption toggle — never set it on a full closed-caption track.

Step 3 — Emit the DASH text AdaptationSet with lxml

DASH declares the caption track inside the MPD rather than a sidecar playlist. A text AdaptationSet carries contentType="text"; its mimeType is application/mp4 with codecs="stpp" for IMSC1 fragmented into fMP4 (CMAF), or text/vtt for raw sidecar WebVTT. A Role descriptor marks it as a subtitle track and a SegmentTemplate describes the fragment naming.

from lxml import etree

MPD_NS = "urn:mpeg:dash:schema:mpd:2011"

def build_text_adaptation_set(period, *, lang: str, rep_id: str,
                              seg_seconds: float = SEG_SECONDS,
                              timescale: int = 90_000, imsc1: bool = True):
    q = lambda tag: etree.SubElement(period if tag == "AdaptationSet" else aset, tag)
    # contentType="text" is what marks this as a timed-text AdaptationSet.
    aset = etree.SubElement(period, "AdaptationSet",
                            contentType="text", lang=lang, segmentAlignment="true")
    if imsc1:
        aset.set("mimeType", "application/mp4")   # IMSC1 in fMP4 / CMAF
        codecs = "stpp"                            # TTML/IMSC1 sample entry
    else:
        aset.set("mimeType", "text/vtt")          # raw sidecar WebVTT
        codecs = "wvtt"
    # DASH role descriptor — without it the track exists but is not offered as a subtitle.
    etree.SubElement(aset, "Role",
                     schemeIdUri="urn:mpeg:dash:role:2011", value="subtitle")
    rep = etree.SubElement(aset, "Representation",
                           id=rep_id, bandwidth="2000", codecs=codecs)
    # timescale in 90 kHz ticks; duration in those ticks keeps DASH aligned to the video clock.
    etree.SubElement(rep, "SegmentTemplate",
                     media="text_$Number$.m4s", initialization="text_init.mp4",
                     timescale=str(timescale),
                     duration=str(int(seg_seconds * timescale)), startNumber="1")
    return aset

def build_mpd(lang: str = "en", rep_id: str = "text-en") -> bytes:
    mpd = etree.Element("MPD", nsmap={None: MPD_NS},
                        type="static", profiles="urn:mpeg:dash:profile:isoff-on-demand:2011",
                        minBufferTime="PT2S")
    period = etree.SubElement(mpd, "Period", id="0")
    build_text_adaptation_set(period, lang=lang, rep_id=rep_id)
    return etree.tostring(mpd, pretty_print=True,
                          xml_declaration=True, encoding="UTF-8")

The timescale="90000" in the SegmentTemplate deliberately matches the HLS 90 kHz clock, so a single SEG_SECONDS value produces byte-identical segment boundaries on both branches. Mixing timescales (for example 1000 on text but 90000 on video) is the most common source of DASH caption drift, because the duration integer then rounds differently on each track.

The codecs choice is not cosmetic. stpp (the TTML/IMSC1 sample entry defined in ISO 14496-30) tells the player to expect fragmented IMSC1 samples wrapped in fMP4, which is the CMAF-compatible packaging that lets one text fragment set serve both HLS and DASH from a single origin. wvtt signals raw WebVTT in fMP4 instead. Advertising one while serving the other is a hard decode failure on strict players, so the imsc1 flag here must track whatever the packager actually wrote. When the delivery target is CMAF — a single fragmented container consumed by both an HLS #EXT-X-MAP and a DASH SegmentTemplate — wrap the timed text as IMSC1 in fMP4 and keep the fragment duration identical to the video fragments so the two protocols share one physical segment set on the CDN.

Threshold Reference Table

Every numeric constant the segmenter and manifests depend on, with its source.

Verification & Test Pattern

Validate structurally before you publish: parse the generated media playlist back and assert #EXT-X-TARGETDURATION is not less than any #EXTINF, and assert every segment file opens with a WEBVTT line followed by an X-TIMESTAMP-MAP. In a full pipeline this is the concept Apple’s mediastreamvalidator enforces; the check below is the fast unit-test proxy that runs before it.

from m3u8 import load as m3u8_load

def verify_media_playlist(path: str):
    pl = m3u8_load(path)
    assert pl.is_endlist, "VOD playlist must end with #EXT-X-ENDLIST"
    longest = max(seg.duration for seg in pl.segments)
    # RFC 8216 §4.4.3.1 — EXT-X-TARGETDURATION must be >= every EXTINF (rounded up)
    assert pl.target_duration >= round(longest), "TARGETDURATION below a segment duration"

def verify_segment_headers(seg_paths):
    for p in seg_paths:
        with open(p, encoding="utf-8") as fh:
            head = fh.read(120)
        assert head.startswith("WEBVTT"), f"{p} missing WEBVTT signature"
        # A missing X-TIMESTAMP-MAP makes the player assume LOCAL zero == media zero
        assert "X-TIMESTAMP-MAP=MPEGTS:" in head, f"{p} missing time-map"

For DASH, validate the MPD against the ISO 23009-1 schema and assert the text AdaptationSet carries both contentType="text" and a child Role with value="subtitle" — the two attributes a player uses to list the track in its subtitle menu.

Troubleshooting / Failure Modes

Captions drift steadily later as the program runs : Root cause: the X-TIMESTAMP-MAP MPEGTS value does not match the media’s actual presentation offset, so every cue is shifted by a constant that reads as growing drift against a moving picture. Fix: read the first video PTS from the packaged media and set MPEGTS: to that exact tick value instead of assuming 900000; verify against the video with mediastreamvalidator.

Caption track is present in the manifest but never appears in the player menu : Root cause (HLS): the #EXT-X-MEDIA GROUP-ID is not referenced by any #EXT-X-STREAM-INF SUBTITLES= attribute, so no variant advertises it. Root cause (DASH): the text AdaptationSet has no child Role descriptor. Fix: add the SUBTITLES="<group-id>" attribute to every variant, or add <Role schemeIdUri="urn:mpeg:dash:role:2011" value="subtitle"/>.

Captions load but are off by exactly ten seconds : Root cause: the segmenter wrote MPEGTS:900000 (the 10 s Apple offset) but the media was packaged with a zero start offset, so the caption timeline is 10 seconds ahead. Fix: set PTS_OFFSET to match the packager — either both use 900000 or both use 0.

Player picks the wrong language or forces a subtitle on : Root cause: LANGUAGE uses a three-letter ISO 639-2 code the player does not map, or FORCED=YES was set on a full caption track. Fix: convert to a BCP-47 tag and reserve FORCED=YES for narrative foreign-dialogue renditions only.

DASH captions drift while HLS from the same source is fine : Root cause: the SegmentTemplate timescale differs from the media timescale, so integer duration ticks round to a boundary the video does not share. Fix: set timescale="90000" on the text AdaptationSet to match the media clock and recompute duration as seg_seconds × 90000.

A cue disappears exactly at a segment boundary : Root cause: a boundary-spanning cue was assigned to only one segment, so it vanishes when the player switches to the adjacent segment mid-display. Fix: duplicate any cue whose interval overlaps a segment into every segment it touches, as the membership test in Step 1 does.

Operational Notes

At scale the segmenter is CPU-cheap but I/O-heavy: a feature with a dozen language tracks at 6-second segments emits thousands of tiny files, and the CDN cache-key explosion, not the segmentation, becomes the bottleneck. Batch the per-language segmentation and emit segments to object storage directly rather than staging on local disk, following the bounded-worker I/O pattern in async batch caption processing. Keep the IMSC1 packaging decision — raw sidecar WebVTT versus fMP4-wrapped IMSC1 — consistent with the OTT container choice in IMSC1 & TTML packaging for OTT, because a DASH AdaptationSet that advertises codecs="stpp" but serves raw WebVTT will fail decode on strict players.

Cache the manifests with a short TTL and the segments with a long one: segment bytes are immutable once written, but a manifest edit (adding a language, flipping a DEFAULT) must propagate quickly. Retain the source WebVTT and the generated segment set together so a re-segmentation at a different target duration is reproducible from the same authored cues.

Related

• IMSC1 & TTML packaging for OTT — the fMP4/CMAF text container that DASH codecs="stpp" fragments come from.
• Embedding CEA-608 & CEA-708 in transport streams — the in-band alternative to sidecar text delivery.
• WebVTT segmentation for HLS — per-segment X-TIMESTAMP-MAP math, boundary duplication and PTS rollover in depth.
• WebVTT cue extraction & validation — the cue-list QC this segmenter assumes upstream.
• Async batch caption processing — bounded-worker I/O for segmenting many language tracks in parallel.

Part of: Caption Muxing, Packaging & Delivery — the container, mux and delivery reference for broadcast and OTT caption tracks.