WebVTT Cue Extraction & Validation

WebVTT is structurally permissive in a way that breaks naive parsers: a single .vtt file can carry a BOM, a NOTE block, STYLE and REGION headers, optional cue identifiers, and inline <c>/<v> markup — none of which a downstream encoder or compliance auditor wants in the text payload. The engineering gap this page closes is the deterministic reduction of that loose syntax to a uniform, frame-checkable cue model and the validation of every cue against hard numeric limits before it reaches mux or playout: a maximum reading speed of 180 WPM live / 150 WPM pre-recorded (≈ 17 characters per second), a temporal overlap tolerance of ≤ 50 ms between consecutive cues, and a structural cap of 32 characters per line × 2 lines per cue. Miss any of those and you ship cue collisions, on-screen flicker, or reading-rate failures that surface as reportable defects in an FCC or Ofcom audit.

This step is one stage in the broader SRT, SCC & WebVTT parsing workflows reference. Where parsing SCC with Python libraries reconstructs CEA-608 state from hex byte-pairs and SRT timestamp normalization quantizes plaintext cues to a frame grid, the WebVTT lane has the opposite problem: too much structure, much of it optional, all of it a place for a malformed file to hide. The validators below read every limit from the threshold reference table so the same code re-points at a different jurisdiction by swapping a config row.

Pipeline stage & prerequisites

Cue extraction and validation runs at the post-ingest, pre-normalization boundary: after a .vtt asset arrives from an NLE export, an ASR service, or an OTT packager, and before its cues are mapped to a program timeline or transcoded to CEA-608/708 ancillary data. Placing the gate here is what keeps it cheap — a 33-character line caught at ingest is a re-wrap; the same line caught after mux is a re-render of the whole package.

The validators assume nothing about upstream cleanliness — that is the point of the header sanitization step. They produce a uniform {index, start_ms, end_ms, text} record that the next stage consumes without re-checking. Continuous sync measurement against a live audio reference is out of scope here and is handled by automated sync drift detection; this page enforces the static, per-cue acceptance criteria. Reading-rate enforcement shares its threshold config with enforcing character rate limits in QC so the CPS ceiling is defined once.

Required tooling:

Step-by-step implementation

The implementation is a four-step pipeline: sanitize the header, parse cues to a normalized model, run the threshold gates statelessly, then fold the result into a routing verdict. Every step is a pure function so quarantined assets can be re-run after a fix without special-casing.

Step 1 — Sanitize the header and verify the magic string

The W3C WebVTT specification requires the file to begin with the string WEBVTT, optionally preceded by a UTF-8 BOM (U+FEFF) and optionally followed by a space/tab and a header label. Vendor exports routinely violate the surrounding rules — a stray BOM counted as content, CRLF line endings, or a NOTE block jammed against the magic string. Detect the encoding and strip the BOM before handing bytes to the parser, or webvtt-py will see WEBVTT and reject the file.

from charset_normalizer import from_bytes

def sanitize_header(path: str) -> str:
    """Return decoded VTT text with BOM stripped and CRLF normalized.

    W3C WebVTT §4.1 — a file MUST begin with an optional U+FEFF BOM
    followed by the ASCII string "WEBVTT".
    """
    raw = open(path, "rb").read()
    best = from_bytes(raw).best()              # detect cp1252/utf-16/utf-8 exports
    text = str(best) if best else raw.decode("utf-8", errors="replace")
    text = text.lstrip("")               # strip BOM counted as content
    text = text.replace("\r\n", "\n").replace("\r", "\n")  # normalize line endings
    first = text.split("\n", 1)[0].strip()
    if not first.startswith("WEBVTT"):         # §4.1 signature gate
        raise ValueError(f"Not a WebVTT file: first line is {first!r}")
    return text

Step 2 — Parse cues to a normalized millisecond model

webvtt-py returns each cue as a Caption object whose start and end are strings in the HH:MM:SS.mmm (or MM:SS.mmm) grammar defined by the spec. Broadcast arithmetic needs a single integer representation, so normalize every timestamp to milliseconds at the parse boundary and never compute on the string form. Parse from the sanitized buffer rather than the file path so Step 1’s cleanup is not bypassed.

import io
from dataclasses import dataclass
import webvtt

@dataclass
class Cue:
    index: int
    start_ms: int
    end_ms: int
    text: str

def _ts_to_ms(value: str) -> int:
    """Normalize a WebVTT timestamp to integer milliseconds.

    W3C WebVTT §4.3 — timestamp is [HH:]MM:SS.mmm; minutes/seconds are
    two digits, fraction is exactly three digits.
    """
    h, m, rest = ("00", *value.split(":"))[-3:]   # tolerate MM:SS.mmm
    s, _, frac = rest.partition(".")
    ms = (frac + "000")[:3]                        # pad/truncate to 3 digits
    return ((int(h) * 60 + int(m)) * 60 + int(s)) * 1000 + int(ms)

def parse_cues(vtt_text: str) -> list[Cue]:
    captions = webvtt.read_buffer(io.StringIO(vtt_text))   # skips NOTE/STYLE/REGION
    cues = []
    for i, cap in enumerate(captions):
        start, end = _ts_to_ms(cap.start), _ts_to_ms(cap.end)
        if end <= start:                            # zero/negative duration is invalid
            raise ValueError(f"cue {i}: end {cap.end} <= start {cap.start}")
        cues.append(Cue(i, start, end, cap.text.strip()))
    return cues

Step 3 — Run the stateless validation gates

Each cue is checked against three independent dimensions: reading speed, structural geometry, and temporal relationship to its neighbour. Keep the function pure — it reads cues and the threshold config, mutates nothing, and returns a violation list. Reading speed is computed in characters per second (the metric CEA-708 decoders and Ofcom both use) and cross-checked against the words-per-minute ceiling.

# Thresholds — see the reference table; sourced from FCC §79.1 / Ofcom / W3C
MAX_CPS            = 17     # Ofcom Annex 1 — ~17 chars/sec sustained reading speed
MAX_WPM            = 180    # FCC §79.1(j) live ceiling; pre-recorded target 150
MAX_CHARS_PER_LINE = 32     # ANSI/CTA-608-E — 32-column caption grid
MAX_LINES_PER_CUE  = 2      # EBU-TT-D / Ofcom — two-line on-screen maximum
MAX_OVERLAP_MS     = 50     # consecutive cues must not overlap beyond decoder flush window
MAX_GAP_MS         = 2000   # flag suspicious silence between cues for review

def validate_cue(cue: Cue, prev: Cue | None) -> list[str]:
    v = []
    dur_s = (cue.end_ms - cue.start_ms) / 1000.0
    chars = len(cue.text.replace("\n", ""))

    cps = chars / dur_s if dur_s > 0 else float("inf")
    if cps > MAX_CPS:                                       # reading-speed gate
        v.append(f"reading speed {cps:.1f} CPS > {MAX_CPS}")
    wpm = (len(cue.text.split()) / dur_s) * 60 if dur_s > 0 else float("inf")
    if wpm > MAX_WPM:
        v.append(f"reading speed {wpm:.0f} WPM > {MAX_WPM}")

    lines = [ln for ln in cue.text.splitlines() if ln.strip()]
    if len(lines) > MAX_LINES_PER_CUE:                     # structural geometry
        v.append(f"{len(lines)} lines > {MAX_LINES_PER_CUE}")
    if any(len(ln) > MAX_CHARS_PER_LINE for ln in lines):
        v.append(f"line width > {MAX_CHARS_PER_LINE} chars")

    if prev is not None:                                   # temporal relationship
        delta = cue.start_ms - prev.end_ms
        if delta < -MAX_OVERLAP_MS:
            v.append(f"overlap {abs(delta)}ms > {MAX_OVERLAP_MS}ms")
        elif delta > MAX_GAP_MS:
            v.append(f"gap {delta}ms > {MAX_GAP_MS}ms")
    return v

Step 4 — Fold cues into a routing verdict

The per-cue results aggregate into one machine-readable record that the media asset management (MAM) system consumes over REST. A single violating cue sets pipeline_action to quarantine and attaches the failing cue index and timecode for the audit trail.

import json

def build_verdict(asset_id: str, cues: list[Cue]) -> str:
    violations, prev = [], None
    for cue in cues:
        cv = validate_cue(cue, prev)
        if cv:
            violations.append({"cue": cue.index,
                               "start_ms": cue.start_ms,
                               "issues": cv})
        prev = cue
    passed = not violations
    return json.dumps({
        "asset_id": asset_id,
        "cue_count": len(cues),
        "compliance_status": "PASS" if passed else "FAIL",
        "violations": violations,
        "pipeline_action": "ROUTE_TO_TIMELINE" if passed else "QUARANTINE",
    }, indent=2)

A PASS verdict hands the normalized cue list to the timeline-alignment stage; the offset correction and frame-accurate rounding that step performs are covered in mapping WebVTT cues to broadcast timelines. When the same cues must be re-emitted as SRT or SCC for archival or 608 carriage, the millisecond model produced here is the input to SRT timestamp normalization, which quantizes the boundaries onto a drop-frame or non-drop-frame grid.

Threshold reference table

Every limit the validators read lives here, not in prose, so a config swap re-points the same code at a different jurisdiction or timebase.

Verification & test pattern

Lock each threshold behind a pytest fixture so a config change that loosens a gate fails CI rather than silently shipping a non-compliant asset. Test both sides of every boundary — exactly at the limit must pass, one unit beyond must fail.

import pytest
from validator import Cue, validate_cue, _ts_to_ms, MAX_OVERLAP_MS

def test_timestamp_normalization():
    assert _ts_to_ms("01:02:03.456") == 3_723_456   # W3C §4.3 HH:MM:SS.mmm
    assert _ts_to_ms("02:03.4") == 123_400           # MM:SS.m tolerated + padded

def test_line_width_boundary():
    ok  = Cue(0, 0, 2000, "X" * 32)                  # ANSI/CTA-608-E grid width
    bad = Cue(1, 0, 2000, "X" * 33)
    assert validate_cue(ok, None) == []
    assert any("line width" in m for m in validate_cue(bad, None))

def test_overlap_just_over_tolerance_fails():
    prev = Cue(0, 0, 2000, "hello")
    # next cue starts 51 ms before prev ends -> exceeds 50 ms flush window
    nxt = Cue(1, 2000 - (MAX_OVERLAP_MS + 1), 4000, "world")
    assert any("overlap" in m for m in validate_cue(nxt, prev))

def test_reading_speed_flags_dense_cue():
    cue = Cue(0, 0, 1000, "supercalifragilistic")    # 20 chars in 1.0 s = 20 CPS
    assert any("CPS" in m for m in validate_cue(cue, None))

A useful fixture is a deliberately broken .vtt containing a BOM, a NOTE block, a 33-character line, and a 60 ms overlap; a correct pipeline must strip the first two and flag the last two with the exact cue indices.

Troubleshooting / failure modes

webvtt-py raises MalformedFileError on a file that plays fine : Root cause: a BOM or non-UTF-8 encoding (cp1252 from a Windows NLE) precedes the WEBVTT magic string. Fix: run Step 1’s sanitize_header with charset_normalizer before parsing; never hand the raw path to webvtt.read for vendor exports.

Reading-speed gate passes but captions feel rushed on screen : Root cause: CPS computed on text that still contains inline <c.classname> / <v Speaker> markup, inflating the duration-normalized character count’s denominator or counting tag characters. Fix: strip cue markup to plain text before measuring; only displayed glyphs count toward the 17 CPS ceiling.

Overlap violations fire on every cue in a roll-up-style file : Root cause: the source intentionally overlaps cues for a paint-on effect, or end times were authored as the next cue’s start. Fix: distinguish authored overlap from drift by checking whether the overlap is constant; if it is a deliberate style, relax MAX_OVERLAP_MS for that asset class via config rather than in code.

Timestamps drift by whole seconds after NLE export : Root cause: a frame-rate conversion (25 fps timeline exported against a 23.976 fps master) applied during export, producing cumulative not constant error. Fix: this is a timeline problem, not a parse problem — hand the normalized cues to mapping WebVTT cues to broadcast timelines for offset and frame-rate correction.

Line-count gate misses a long single line that wraps to three rows on the decoder : Root cause: the cue has one logical line over 32 characters that the renderer wraps; splitlines() sees one line, the screen shows three. Fix: validate effective rows by simulating a 32-column word wrap, not just counting explicit \n breaks.

Duplicate or empty cue identifiers crash a downstream serializer : Root cause: WebVTT cue identifiers are optional and need not be unique; code that keys a dict on cap.identifier collides or drops cues. Fix: key on positional index, treat the identifier as opaque metadata only.

Operational notes

At batch scale this stage is CPU-light and I/O-bound: the cost is reading and decoding files, not the arithmetic. Stream cues from webvtt.read_buffer and validate incrementally rather than materializing every Caption object — a feature-length program is tens of thousands of cues, and only the previous cue is needed for the overlap check, so memory stays flat regardless of runtime. Keep per-worker resident size bounded by holding one cue plus its predecessor, never the whole list, when you only need the verdict.

Size the worker pool to roughly the number of physical cores; because the work is short and file-bound, a process pool around the decode call scales better than threads fighting the GIL. For very large libraries, route this stage through the shared async batch caption processing runner so WebVTT, SCC, and SRT assets share one quarantine queue and backpressure policy. Keep every validator stateless and idempotent — the same asset must always yield the same verdict — and write each verdict, pass or fail, to an append-only store keyed by asset_id and content hash so the compliance trail survives the asset.

Related

• Parsing SCC with Python libraries — CEA-608 state-machine decoder for the byte-level sibling format.
• SRT timestamp normalization — Frame-grid quantizer that consumes the millisecond cue model produced here.
• Mapping WebVTT cues to broadcast timelines — Offset and frame-rate correction applied to validated cues.
• Async batch caption processing — Worker-pool runner and quarantine queue this stage plugs into at scale.
• FCC Part 79 compliance checklist — The acceptance gates a passing cue set must satisfy before mux.

Part of: SRT, SCC & WebVTT Parsing Workflows.