[Artifact Forest 5/9] Recover screenplay structure from PDF and unstructured text #426

Merged implementation — 2026-10-13

PR #446 merged at 8e0f5f4 with maintainer authorization. The bounded PDF/text recovery implementation is on main, including explicit optional OCR, retained evidence, and lossy/review-required native body derivatives.

#429 is reconciling the recovery provider inventory, doctor schema, setup guidance and deferred qualification record. Tests, builds, lint, native probes/conversions, schema validation and CI remain deferred. This issue stays open and its acceptance boxes stay unchecked for the later deep testing session.

Current handoff: docs/work/426/HANDOFF.md. Earlier draft status below is historical.

Parent: #421
Depends on: #423
Coordinates with: #424, #425

Outcome

Support an explicitly lossy, review-required path from screenplay-shaped PDF or text into a recoverable screenplay representation and then into Fountain/other script derivatives.

Pipeline

PDF
 ├─ text-bearing → deterministic text/layout extraction
 └─ image-only   → bounded OCR fallback
                     ↓
              screenplay recovery
                     ↓
          renderflow.screenplay/v1
             ├─ Fountain
             ├─ FDX
             ├─ FadeIn
             └─ OSF

Provider candidates

• Poppler pdftotext / equivalent local deterministic extractor.
• Existing Tesseract provider for image-only pages.
• Optional reviewed AI transform only when deterministic recovery is insufficient and policy explicitly allows it.

Requirements

• Detect whether PDF has usable text before OCR.
• Preserve page boundaries and extraction evidence.
• Define deterministic screenplay heuristics for scene headings, character cues, dialogue, parentheticals, and action.
• Emit confidence/ambiguity diagnostics per recovered element.
• Never silently treat OCR or heuristic output as the original screenplay.
• AI recovery must be optional, separately evidenced, and never required for deterministic extraction.
• Feed the recovered screenplay through [Artifact Forest 2/9] Define canonical screenplay artifacts and capability truth #423 validation before downstream conversions.
• Do not regenerate PDF by default when PDF was the source; source-format suppression from [Artifact Forest 1/9] Define cycle-safe exhaustive derivative planning #422 applies.

Acceptance criteria

• Text PDF → reviewed screenplay candidate works on a synthetic fixture.
• Image-only PDF takes the OCR path or reports provider unavailability.
• Recovery evidence names extraction method and confidence/ambiguity.
• Fountain/FDX/etc. derivatives are marked as recovered/lossy lineage.
• Source PDF is unchanged and not regenerated by an exhaustive forest unless explicitly requested.
• Malformed/encrypted/unsupported PDFs produce typed outcomes.
• Optional AI cannot upgrade confidence or approval without explicit evidence/review.

Non-goals

Claiming exact Final Draft reconstruction or visual-layout preservation from arbitrary PDFs.

Authored checkpoint — draft PR #446

The #426 implementation is available in draft PR #446, based on main after #445 was merged.

Authored: bounded PDF/text extraction with explicitly pinned optional OCR; deterministic canonical recovery with complete page/line evidence; Fountain/FDX/Fade In/OSF body exports; CLI and SDK planning/publication; mandatory lossy/review-required lineage; schemas, documentation, examples, and deferred regression cases. No AI fallback or source-PDF regeneration route is supplied.

Per the sprint instruction, tests, compilation/builds, lint/formatting, schema validation, native probes/conversions, installations/downloads, and CI were not run. Verification was source review plus exact Git content identity. All acceptance checkboxes remain unqualified and this issue stays open for the later testing session.

Handoff: docs/work/426/HANDOFF.md.

Audit-to-roadmap reconciliation — 2026-10-21

Audit: https://github.com/egohygiene/renderflow/blob/audit/flow-suite-20261021T025401Z/audits/flow-suite-20261021T025401Z.md · Complete roadmap: #367

PDF/text screenplay recovery authored; optional explicit OCR; lossy review-required lineage, no implicit source PDF regeneration.

Bounded preparation: Review ambiguity/refusal and lineage cases; prepare bounded qualification records.

Full outcome remains gated (*): Exact Poppler/OCR assets and deferred recovery/native export/schema/runtime human review.

The maintainer's implementation-sprint qualification deferral remains in force. This audit ran no Renderflow tests, compiler/build/Cargo resolution, product lint/format/schema qualification, native/CLI/provider/visual execution, installers or new CI. Reading existing exact-source provider records is not a qualification rerun. Source implementation and authored tests are not passing-test, native-fidelity, release or downstream-consumer evidence.

This is an additive planning note. The full original issue, acceptance criteria, scope, parent references and historical checkpoints remain authoritative and unchanged. No acceptance box or issue is closed by this audit. Recommended ledger parent: #421.