A Rust WHATWG HTML5 parser: tokenizer + full tree construction, transcribed directly from the spec rather than ported from another implementation.
- Spec-derived, not guessed. Every state and algorithm is transcribed
from the raw WHATWG HTML parsing specification text — adoption agency,
foster parenting, foreign content (SVG/MathML), frameset,
<template>'s real content-fragment model, all included. - 100% conformant against the html5lib-tests tree-construction corpus — 1,726/1,726 applicable cases pass (see Testing below).
- Per-node source positions. Every node in the resulting tree carries its line/column/byte offset in the original input, not just the parsed structure — useful for anything that needs to point back at source (linters, validators, diagnostics).
- Parse errors, not just a tree.
parse()also reports every WHATWG "parse error" (§13.2.2) encountered, tokenizer- and tree-construction- level alike, each with its own source position — useful for diagnostics tooling, not just silent best-effort recovery. - No dependencies, no
unsafe.
use html5_parser::{parse, Document, NodeId, NodeKind};
fn main() {
let result = parse("<!DOCTYPE html><title>Hi</title><h1>Hello, world!</h1>");
print_tree(&result.document, result.document.root(), 0);
for error in &result.errors {
println!("{:?} at {:?}", error.kind, error.position);
}
}
fn print_tree(document: &Document, node: NodeId, depth: usize) {
let indent = " ".repeat(depth);
match &document.node(node).kind {
NodeKind::Element { name, .. } => println!("{indent}<{name}>"),
NodeKind::Text { content } => println!("{indent}{content:?}"),
_ => {}
}
for child in document.children(node) {
print_tree(document, child, depth + 1);
}
}Document's read-only API (root/node/parent/children) is
intentionally minimal — just enough to walk the tree and read each
node's kind and source position. See docs.rs
for the full API.
Two narrow, deliberately out-of-scope sub-features, neither exercised by the html5lib-tests corpus:
<template>'s classic content-fragment model (a real, inertNodeKind::DocumentFragmentper template element) is implemented, but two much newer, still-evolving sub-features layered onto<template>in the current spec are not: declarative shadow DOM (shadowrootmodeand friends) and content patching (theforattribute) — both would require modeling shadow roots/custom element registries this crate has no other use for.<selectedcontent>'s option-mirroring (the "customizable<select>" proposal) is implemented for ordinary, non-scripted parse-time use, but simplifies a few edge cases: nomultiple<select>support, and a practically- rather than fully-generally- scoped "list of options" walk (doesn't handle every possible<optgroup>nesting shape).
Besides hand-written unit/end-to-end tests (src/tokenizer.rs,
src/tree_builder.rs, src/lib.rs, src/document.rs), cargo test
also runs tests/html5lib_conformance.rs: every applicable case
(full-document, non-fragment, non-scripting — see
tests/html5lib-tests/README.md) from the vendored
html5lib-tests
tree-construction corpus — currently all 1,726 applicable cases pass
(100%). tests/html5lib_known_failures.txt is currently empty (kept,
rather than deleted, as the harness's regression-tracking mechanism —
see that file's header — for whenever a future corpus refresh or code
change introduces a real one).
Implementation decisions are derived from the
WHATWG HTML parsing specification.
Other implementations (e.g. html5ever) are explanatory references only,
not a source to copy code from.
HTML input (string) → tokenizer (WHATWG tokenizer state machine)
→ tree_builder (WHATWG tree-construction algorithm,
incl. foreign content / SVG / MathML)
→ document (element/text/comment tree with positions)
This crate started as a staged, sibling-project effort: build only what
html-conform needed to
replace its previous HTML5-parsing dependency, prove that against real
usage, and only then decide on a public API and whether to publish
standalone. In practice, full spec coverage and a first crates.io
publish happened ahead of that cross-repo validation step, on direct
request — see plan/DECISIONS.md for the decision log and plan/
generally for the phase-by-phase implementation history (not tracked in
git, see CLAUDE.md).
MIT — see LICENSE.