Count em-dash HTML entities in em-dash-overuse

The em-dash-overuse text analyzer ran stripHtmlToText over raw markup,
which drops tags but leaves character entities intact. A model that wrote
—, —, or — rendered a real em-dash the counter never
saw, so 12 entity-escaped dashes on a live page slipped through.

Decode the em-dash entities (named, zero-padded decimal, upper/lower hex)
to the literal glyph before counting. En-dash entities stay untouched: the
rule counts em-dashes, and the literal en-dash was never counted either.

The gap lived only in the regex / static-HTML path (detectText and
detect-html's runTextContentAnalyzers, both over raw HTML). The browser
adapter never ran this analyzer, so build:browser and build:extension
produce no diff.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Paul Bakaus
2026-07-22 00:27:59 -07:00
co-authored by Claude Fable 5
parent 0376145a46
commit 7dcca2bb36
3 changed files with 103 additions and 1 deletions
+9 -1
View File
@@ -309,8 +309,16 @@ const REGEX_ANALYZERS = [
// Em-dash overuse: 5+ em-dashes or "--" in body text content
// (occasional em-dash use in prose is fine; the pattern fires only
// when count crosses into AI-cadence territory).
//
// stripHtmlToText drops tags but leaves character-entity escapes intact, so
// a model that writes `&mdash;`, `&#8212;`, or `&#x2014;` renders an em-dash
// the counter never saw. Decode the em-dash entities (named, zero-padded
// decimal, upper/lower hex) to the literal glyph first. En-dash entities are
// deliberately left alone: the rule counts em-dashes, and the literal ``
// was never counted either.
(content, filePath) => {
const text = stripHtmlToText(content);
const text = stripHtmlToText(content)
.replace(/&mdash;|&#0*8212;|&#x0*2014;/gi, '—');
let count = 0;
const re = /[—]|--(?=\S)/g;
while (re.exec(text) !== null) count++;