Specs: text-extraction-eml (#1438)#1495
Conversation
Adds the spec set for extending TextExtractionService with EML support via zbateson/mail-mime-parser. Two outputs share the parser: - Flat plain-text extraction (headers + body + recursively-extracted attachment text inline) for entity detection. - A new public parseEmlStructured() returning headers + both text/plain and text/html body parts + attachments-with-bytes (recursive nestedEml capped at depth 3) for downstream PDF assembly. Tightly scoped: does not retrofit the broader TextExtractionService surface (currently uncovered by an OpenSpec capability). Refs: #1438 Pair: ConductionNL/docudesk#135 (eml-pdf-assembly hard dep, anonymise-output-as-pdf-by-default soft dep).
Quality Report — ConductionNL/openregister @
|
| Check | PHP | Vue | Security | License | Tests |
|---|---|---|---|---|---|
| lint | ✅ | ||||
| phpcs | ✅ | ||||
| phpmd | ✅ | ||||
| psalm | ✅ | ||||
| phpstan | ✅ | ||||
| phpmetrics | ✅ | ||||
| eslint | ❌ | ||||
| stylelint | ✅ | ||||
| composer | ✅ | ✅ 153/153 | |||
| npm | ✅ | ✅ 598/598 | |||
| PHPUnit | ❌ | ||||
| Newman | ⏭️ | ||||
| Playwright | ⏭️ |
Quality workflow — 2026-05-12 09:22 UTC
Download the full PDF report from the workflow artifacts.
WilcoLouwerse
left a comment
There was a problem hiding this comment.
(inline-only comment review — verdict follows in a separate REQUEST_CHANGES review)
WilcoLouwerse
left a comment
There was a problem hiding this comment.
1 blocker requires a fix — the content encoding contract is ambiguous and will corrupt DocuDesk PDF/A-3 attachments. Five concerns (body-as-array vs value-object, MAY throw vs downstream contract, missing multipart/alternative scenario, PII logging not specified, and spec format vs writing-specs.md) — the format concern is repo-wide drift, surfaced here for visibility rather than as a PR blocker. Three minor polish suggestions. Spec design overall is solid: capability scope is tightly bounded, the two-path architecture (flat + structured) is well-justified, scenarios are comprehensive for the headers/recursion paths. CI Vue Quality (eslint) is failing but unrelated to this spec-only PR (no Vue files touched).
Blocker: - B1: tighten EmlAttachment.content contract — MUST hold decoded binary bytes (via $part->getContent(), not getRawContent()); consumers MUST NOT base64-decode again. Prevents downstream PDF/A-3 attachment corruption in DocuDesk's eml-pdf-assembly. Concerns: - C1: fix scenario notation — body accessors switched from array form result.body['plainText'] to value-object form result.body->plainText to match the typed EmlBody declaration. - C2: parseEmlStructured changed from MAY throw to MUST throw on irrecoverable malformed input — so DocuDesk's exception-driven fallback (eml-pdf-assembly → extractEml flat text) actually fires. - C3: add multipart/alternative scenario for the flat path — confirms text/plain is preferred over text/html when both are present. - C4: add no-PII-in-logs Requirement (ADR-005) with two Scenarios — parser-failure log lines and sanitised exception-message logging MUST NOT leak addresses / names / subjects / body content / attachment filenames. Minors: - M1: clarify depth-3 recursion semantics — depth is measured as the number of recursive parseEmlStructured calls from the root (root = depth 0, limit of 3 allows parses at depths 0, 1, 2, 3). - M2: add SHOULD Requirement + Scenario for non-UTF-8 body charset transcoding (mb_detect_encoding + mb_convert_encoding to UTF-8). - M3: state the missing-filename fallback format (attachment-<n>, 1-indexed) on EmlAttachment, with a new Scenario covering it. Tasks: update 3.4 (filename fallback + decoded-bytes), 3.9 (MUST throw), and add 3.10 (PII-sanitised logging) + 3.11 (explicit multipart/alternative preference). Add tests 5.6 (multipart pref), 5.7 (filename fallback), 5.8 (content encoding), 5.9 (PII log redaction), 5.10 (MUST throw assertion). Skipped: C5 (spec-format vs writing-specs.md) — repo-wide drift that Wilco explicitly says does not block merge and needs maintainer coordination.
|
@WilcoLouwerse — thanks for the thorough review. Addressed in Blocker
Concerns
Minors
Skipped (intentionally)
Test coverage 5.6–5.10 added for multipart preference, filename fallback, |
WilcoLouwerse
left a comment
There was a problem hiding this comment.
All 9 prior findings resolved in dabba58 — B1 content-encoding, C1 body notation, C2 MUST throw, C3 multipart scenario, C4 PII/ADR-005, C5 deferred, M1 depth labels, M2 non-UTF-8 SHOULD, M3 filename fallback. Spec is clean; Newman failure is unrelated (docs-only change).
Summary
Adds the OpenSpec change directory for text-extraction-eml — extending
TextExtractionServicewith EML support viazbateson/mail-mime-parser. Two outputs share the parser:parseEmlStructured()returning headers + bothtext/plainandtext/htmlbody parts + attachments-with-bytes (recursivenestedEmlcapped at depth 3) — used for downstream PDF assembly.Tightly scoped — this PR is specs only, no implementation. It also does not retrofit the broader
TextExtractionServicesurface, which is currently uncovered by an OpenSpec capability.Refs: #1438
Pair: ConductionNL/docudesk#135 — DocuDesk's
eml-pdf-assembly(hard dep) andanonymise-output-as-pdf-by-default(soft dep) depend on this.Test plan
openspec/changes/text-extraction-eml/{proposal,design,tasks}.mdand thetext-extraction-eml/spec.mddeltaparseEmlStructured()return shape matches the DocuDesk-side consumer expectations in PR check isset items type #135 (EmlPdfAssemblyService)openspec validate(when the implementation PR lands)