Stage 2b: ingest Clementine + Douay-Rheims psalters into paired verse streams #500

Merged

manlycucumber merged 1 commit from feat/stage2b-psalter-ingest into develop 2026-07-18 14:36:10 +00:00

Conversation 0 Commits 1 Files changed 20 +8104 -21

manlycucumber commented 2026-07-18 14:35:57 +00:00

Owner

Copy link

Stage 2b — real psalter text ingestion (off the calendar)

Vendors both public-domain psalters and wires hermetic, reproducible ingestion into the corpus build (office-text-layer.md §4.2, #112–#114).

• Latin — Clementine Vulgate Project "Quasimodo" Gallican psalter (Ps.lat, Tweedale & Finnegan; released to the public domain — resolves the design's [NEEDS-VERIFICATION] flag). 150 psalms, Ps 151 omitted, Vulgate numbering.
• English — Douay-Rheims 1899, eBible engDRA USFX (Public Domain).

Both vendored under tools/generator/sources/<key>/, sha-pinned in PROVENANCE.json, read with no network by build(). New parsers (parse-clementine.mjs, parse-douay.mjs) → ingestPsalter emit text/clementine-vulgate/psalms.ndjson (2527), text/douay-rheims/psalms.ndjson (2530), and text/versification-report.json.

Faithfulness: every verse is word-for-word faithful to its raw source (token multiset verified against the vendored bytes); ligatures æ/œ/ë intact; both sources clear the born-cited PD-text gate and now appear in MANIFEST.sources. #114 live: the report records the 5 psalms (15, 19, 42, 125, 135) where the editions partition identical text across a different verse count — surfaced, never silently paired.

Validated on workspace (PHP 8.4.23): composer check green (phpcs + phpstan + 1197 phpunit), freeze-golden-year.php zero-diff. Corpus-driven: corpusVersion → 1962-2026-07-18.3, golden unmoved, lock refreshed. Adversarial 3-lens review passed (2 low doc-fidelity findings fixed).

## Stage 2b — real psalter text ingestion (off the calendar) Vendors both public-domain psalters and wires hermetic, reproducible ingestion into the corpus build (`office-text-layer.md` §4.2, #112–#114). - **Latin** — Clementine Vulgate Project "Quasimodo" Gallican psalter (`Ps.lat`, Tweedale & Finnegan; released to the public domain — resolves the design's `[NEEDS-VERIFICATION]` flag). 150 psalms, Ps 151 omitted, Vulgate numbering. - **English** — Douay-Rheims 1899, eBible `engDRA` USFX (Public Domain). Both vendored under `tools/generator/sources/<key>/`, **sha-pinned** in `PROVENANCE.json`, read with **no network** by `build()`. New parsers (`parse-clementine.mjs`, `parse-douay.mjs`) → `ingestPsalter` emit `text/clementine-vulgate/psalms.ndjson` (**2527**), `text/douay-rheims/psalms.ndjson` (**2530**), and `text/versification-report.json`. **Faithfulness:** every verse is word-for-word faithful to its raw source (token multiset verified against the vendored bytes); ligatures `æ/œ/ë` intact; both sources clear the born-cited PD-text gate and now appear in `MANIFEST.sources`. **#114 live:** the report records the 5 psalms (15, 19, 42, 125, 135) where the editions partition identical text across a different verse count — surfaced, never silently paired. **Validated on workspace (PHP 8.4.23):** `composer check` green (phpcs + phpstan + 1197 phpunit), `freeze-golden-year.php` zero-diff. Corpus-driven: `corpusVersion → 1962-2026-07-18.3`, golden unmoved, lock refreshed. Adversarial 3-lens review passed (2 low doc-fidelity findings fixed).

manlycucumber added 1 commit 2026-07-18 14:35:58 +00:00

feat(text): ingest Clementine + Douay-Rheims psalters into paired verse streams (Stage 2b) ... bebb2a4b7b

Vendor the two public-domain psalters and wire hermetic, reproducible ingestion into the
corpus build (office-text-layer.md §4.2, #112-#114).

Sources (vendored under tools/generator/sources/<key>/, sha-pinned in PROVENANCE.json, read
with no network by build()):
- Latin: Clementine Vulgate Project "Quasimodo" Gallican psalter (Ps.lat; ed. Tweedale &
  Finnegan; released to the public domain — resolves the design's [NEEDS-VERIFICATION] flag
  on the digital edition's PD ground). 150 psalms, Ps 151 omitted, Vulgate numbering.
- English: Douay-Rheims 1899, eBible engDRA USFX (Public Domain).

New parsers parse-clementine.mjs (Windows-1252; strips [ ] body brackets, / cola, and the
Ps 118 < > acrostic markers, keeping each Hebrew letter-name as text) and parse-douay.mjs
(USFX book PSA; numbering from the authoritative bcv attribute) feed ingestPsalter, emitting:
- text/clementine-vulgate/psalms.ndjson (2527 verses)
- text/douay-rheims/psalms.ndjson       (2530 verses)
- text/versification-report.json

Every verse is a `full` PD-text leaf cited to its source with an in-edition psalm.verse
locator, so the born-cited gate holds it to a public-domain text source (§4.2c/§4.4); both
sources now appear in MANIFEST.sources. #114 is live: the report records the five psalms
(15, 19, 42, 125, 135) where the two editions partition identical text across a different
verse count — surfaced, never silently paired. Every verse is word-for-word faithful to its
raw source (token multiset verified against the vendored bytes).

Corpus-driven: corpusVersion -> 1962-2026-07-18.3; golden unmoved (no resolver reads
text/** yet); golden provenance lock refreshed. Vendored sources marked -text/binary so
their byte-pin survives checkout.

manlycucumber merged commit d1fb137236 into develop 2026-07-18 14:36:10 +00:00

manlycucumber referenced this pull request from a commit 2026-07-18 14:36:11 +00:00

Merge pull request 'Stage 2b: ingest Clementine + Douay-Rheims psalters into paired verse streams' (#500) from feat/stage2b-psalter-ingest into develop

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels   

area: build

Build & tooling

  

area: ci

CI / release

  

area: compare

Cross-edition / cross-rite comparison tool

  

area: data

Feast corpus & generators

  

area: engine

Resolver / rules / calendar logic

  

area: model

Structured liturgical models (Ordo Missae, Office, diff)

  

area: provenance

Sources, citations, errata, and the resolution trace

  

area: rite

Multi-rite framework and non-Roman rites

  

area: validation

Oracles & day-by-day tests

  

backport

Cherry-pick into a maintained older line

  

blocked

Blocked until something else resolves

  

breaking

Backward-incompatible change (major bump)

  

confirmed

Reproduced / accepted as real

  

dependencies

Pull requests that update a dependency file

  

duplicate

Duplicate of another issue

  

epic

Epic — tracking issue grouping child issues

  

github_actions

Pull requests that update GitHub Actions code

  

invalid

Not a valid issue

  

needs-info

Waiting on more detail

  

needs-triage

New, not yet reviewed

  

php

Pull requests that update php code

  

priority: high

Do next; urgent or high impact

  

priority: low

Nice to have / someday

  

priority: medium

Normal queue

  

regression

Worked in a previous release

  

type: bug

  

type: chore

Maintenance / internal

  

type: correction

A liturgical-data correction backed by a citation

  

type: docs

Documentation

  

type: enhancement

Improvement to existing behaviour

  

type: feature

New user-facing capability

  

type: fix

Bug fix

  

type: perf

Performance improvement

  

type: refactor

Internal refactor

  

type: security

Security fix

  

wontfix

Will not be worked on

No labels

area: build

area: ci

area: compare

area: data

area: engine

area: model

area: provenance

area: rite

area: validation

backport

blocked

breaking

confirmed

dependencies

duplicate

epic

github_actions

invalid

needs-info

needs-triage

php

priority: high

priority: low

priority: medium

regression

type: bug

type: chore

type: correction

type: docs

type: enhancement

type: feature

type: fix

type: perf

type: refactor

type: security

wontfix

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

Directorium/Core!500

No description provided.