diff --git a/README.md b/README.md index ecc24ea3..d2ae383c 100644 --- a/README.md +++ b/README.md @@ -95,19 +95,20 @@ The general file structure for an extension is as follows: - `src/` contains the source code for the extension - `src/main.ts` is the main entry file for the extension (registers commands and wires interlinear XML) - `src/types/interlinearizer.d.ts` is this extension's types file that defines how other extensions can use this extension through the `papi`. It is copied into the build folder - - `src/parsers/interlinearXmlParser.ts` parses interlinear XML into structured data (uses fast-xml-parser). The PT9 XML schema and parsed output are documented in `src/parsers/pt9-xml.md` + - `src/types/` also holds shared enums and type modules (e.g. `interlinearizer-enums.ts`). Use the path alias `types/interlinearizer-enums` in imports instead of relative paths (see `tsconfig.json` paths). + - `src/parsers/` contains all parsers and converters used when importing external data models, one directory per source (e.g. `paratext-9/`). Use the path alias `parsers/...` in imports instead of relative paths (see `tsconfig.json` paths). **Naming:** the program/source is identified only by the directory name (full name, kebab-case). Files inside each directory use generic names (e.g. `converter.ts`, `interlinearParser.ts`, `types.ts`, `lexiconParser.ts`); exported symbols use the full program name (e.g. `Paratext9Parser`, `convertParatext9ToInterlinearization`). - `*.web-view.tsx` files will be treated as React WebViews - `*.web-view.scss` files provide styles for WebViews - `*.web-view.html` files are a conventional way to provide HTML WebViews (no special functionality) - `src/__tests__/` contains unit tests (Jest) for the extension, including parser tests (valid and invalid XML, edge cases) and web-view tests -- `__mocks__/` contains Jest mocks for the PAPI, file modules, and test fixtures used by tests in `src/__tests__/`. The `@papi/backend` and `@papi/frontend` mocks are used mutually exclusively (backend for main.ts tests, frontend for WebView tests); each mock file ends with `export {}` so TypeScript treats it as a module. +- `__mocks__/` (project root) contains Jest mocks for the PAPI, file modules, and test fixtures used by tests in `src/__tests__/`. Manual mocks for the Paratext 9 parser, converter, and lexicon used by `interlinearizer.web-view.test.tsx` live in `src/parsers/paratext-9/__mocks__/` (adjacent to the modules) so that `jest.mock('parsers/paratext-9/...')` picks them up. - `assets/` contains asset files the extension and its WebViews can retrieve using the `papi-extension:` protocol, as well as textual descriptions in various languages. It is copied into the build folder - `assets/displayData.json` contains (optionally) a path to the extension's icon file as well as text for the extension's display name, short summary, and path to the full description file - `assets/descriptions/` contains textual descriptions of the extension in various languages - `assets/descriptions/description-.md` contains a brief description of the extension in the language specified by `` - `contributions/` contains JSON files the platform uses to extend data structures for things like menus and settings. The JSON files are referenced from the manifest - `public/` contains other static files that are copied into the build folder -- `test-data/` contains sample interlinear XML (e.g. `Interlinear_en_MAT.xml`) for development and tests +- `test-data/` contains sample interlinear XML (e.g. `Interlinear_en_JHN.xml`) and Lexicon XML (e.g. `Lexicon.xml`) for development and tests. In tests, resolve paths via `getTestDataPath('Interlinear_en_JHN.xml')` (or `getTestDataPath('Lexicon.xml')`) from `src/__tests__/test-helpers`. Interlinear XML aligns **source** words/WordParses; Lexicon XML is the **target** (gloss) language. Word-level gloss is resolved in the **converter** from Lexicon entries (e.g. `Word:surfaceForm`); the webview Gloss row displays only that data (no surfaceText fallback). - `.github/` contains files to facilitate integration with GitHub - `.github/workflows` contains [GitHub Actions](https://github.com/features/actions) workflows for automating various processes in this repo (e.g. **Test** and **Lint** on push/PR to main, release-prep, hotfix-\*; **Publish** and **Bump Versions** manual dispatch; **CodeQL** for security) - `.github/assets/release-body.md` combined with a generated changelog becomes the body of [releases published using GitHub Actions](#publishing) @@ -118,6 +119,10 @@ The general file structure for an extension is as follows: ## To install +### Requirements + +- **Node.js >= 18** is required. The test suite uses the Web Crypto API (`globalThis.crypto.subtle`) for hashing in the paratext-9 converter tests (e.g. the `sha256HexWebCrypto` path in `src/__tests__/parsers/paratext-9/converter.test.ts` when `convertParatext9ToInterlinearization` is called without the `hashSha256Hex` option). Node 18+ provides this API; older versions will cause those tests to fail. The same requirement is enforced in `package.json` via `engines.node` and is used by CI. + ### Install dependencies: 1. Follow the instructions to install [`paranext-core`](https://github.com/paranext/paranext-core#developer-install). We recommend you clone `paranext-core` in the same parent directory in which you cloned this repository so you do not have to [reconfigure paths](#configure-paths-to-paranext-core-repo) to `paranext-core`. diff --git a/__mocks__/interlinearXmlContent.ts b/__mocks__/interlinearXmlContent.ts index d40e831b..e817ecb5 100644 --- a/__mocks__/interlinearXmlContent.ts +++ b/__mocks__/interlinearXmlContent.ts @@ -1,9 +1,9 @@ /** - * @file Jest mock for webpack ?raw XML import. Exports the contents of test-data/Interlinear_en_MAT.xml + * @file Jest mock for webpack ?raw XML import. Exports the contents of test-data/Interlinear_en_JHN.xml * so interlinearizer.web-view.tsx can parse it in unit tests without webpack. */ import fs from 'fs'; import path from 'path'; -const xmlPath = path.join(__dirname, '..', 'test-data', 'Interlinear_en_MAT.xml'); +const xmlPath = path.join(__dirname, '..', 'test-data', 'Interlinear_en_JHN.xml'); module.exports = fs.readFileSync(xmlPath, 'utf-8'); diff --git a/__mocks__/lexiconXmlContent.ts b/__mocks__/lexiconXmlContent.ts new file mode 100644 index 00000000..8ba7cee9 --- /dev/null +++ b/__mocks__/lexiconXmlContent.ts @@ -0,0 +1,9 @@ +/** + * @file Jest mock for webpack ?raw XML import. Exports the contents of test-data/Lexicon.xml + * so interlinearizer.web-view.tsx can parse it in unit tests without webpack. + */ +import fs from 'fs'; +import path from 'path'; + +const xmlPath = path.join(__dirname, '..', 'test-data', 'Lexicon.xml'); +module.exports = fs.readFileSync(xmlPath, 'utf-8'); diff --git a/cspell.json b/cspell.json index 8760a2f1..fc570e06 100644 --- a/cspell.json +++ b/cspell.json @@ -8,6 +8,7 @@ "node_modules", "package.json", "package-lock.json", + "test-data", "vscode-extension" ], "dictionaryDefinitions": [], @@ -16,7 +17,10 @@ "appdata", "asyncs", "autodocs", + "BCVWP", + "behaviour", "dockbox", + "Eflomal", "electronmon", "endregion", "finalizer", @@ -24,10 +28,14 @@ "guids", "hopkinson", "iframes", + "interlineardata", "interlinearization", + "interlinearizations", "interlinearizer", + "jsmith", "localstorage", "maximizable", + "Morphosyntactic", "networkable", "Newtonsoft", "nodebuffer", @@ -40,6 +48,7 @@ "pdps", "plusplus", "proxied", + "punc", "reinitializing", "reserialized", "sillsdev", @@ -47,15 +56,18 @@ "stringifiable", "Stylesheet", "typedefs", + "unanalyzed", "unregistering", "unregisters", + "unreviewed", "unsub", "unsubs", "unsubscriber", "unsubscribers", "usfm", "verseref", - "versification" + "versification", + "wordform" ], "ignoreWords": [], "import": [] diff --git a/jest.config.ts b/jest.config.ts index 2e43f5fa..591f41b7 100644 --- a/jest.config.ts +++ b/jest.config.ts @@ -27,7 +27,7 @@ const config: Config = { 'src/parsers/**/*.ts', 'src/main.ts', 'src/**/*.web-view.tsx', - '!src/parsers/**/*.d.ts', + '!src/parsers/**/types.ts', '!src/**/__tests__/**', '!src/**/*.test.{ts,tsx}', '!src/**/*.spec.{ts,tsx}', @@ -70,11 +70,12 @@ const config: Config = { */ moduleNameMapper: { /** - * Resolve src-rooted path aliases so tests can use e.g. "@main" or "parsers/..." instead of - * relative paths. Must match tsconfig.json "paths" and webpack resolve.alias. + * Resolve src-rooted path aliases so tests can use e.g. "@main", "parsers/...", or "types/..." + * instead of relative paths. Must match tsconfig.json "paths" and webpack resolve.alias. */ '^@main$': '/src/main', '^parsers/(.*)$': '/src/parsers/$1', + '^types/(.*)$': '/src/types/$1', '\\.(sa|sc|c)ss$': '/__mocks__/styleMock.ts', '\\.(jpg|jpeg|png|gif|eot|otf|webp|svg|ttf|woff|woff2|mp4|webm|wav|mp3|m4a|aac|oga)$': '/__mocks__/fileMock.ts', @@ -93,7 +94,9 @@ const config: Config = { /** Resolve webpack ?inline imports: SCSS content. */ '^(.+)\\.(scss|sass|css)\\?inline$': '/__mocks__/styleInlineMock.ts', /** Resolve webpack ?raw import for test XML in web-view. */ - '^(.+)/Interlinear_en_MAT\\.xml\\?raw$': '/__mocks__/interlinearXmlContent.ts', + '^(.+)/Interlinear_en_JHN\\.xml\\?raw$': '/__mocks__/interlinearXmlContent.ts', + /** Resolve webpack ?raw import for Lexicon XML in web-view. */ + '^(.+)/Lexicon\\.xml\\?raw$': '/__mocks__/lexiconXmlContent.ts', }, /** Exclude dist from module resolution to avoid Haste naming collision with root package.json. */ diff --git a/package-lock.json b/package-lock.json index 080ad78a..af7ddf12 100644 --- a/package-lock.json +++ b/package-lock.json @@ -73,6 +73,9 @@ "webpack-merge": "^6.0.1", "zip-build": "^1.8.0" }, + "engines": { + "node": ">=18" + }, "peerDependencies": { "react": ">=18.3.1", "react-dom": ">=18.3.1" diff --git a/package.json b/package.json index 85207920..8f06319e 100644 --- a/package.json +++ b/package.json @@ -6,6 +6,9 @@ "types": "src/types/interlinearizer.d.ts", "author": "SIL Global", "license": "MIT", + "engines": { + "node": ">=18" + }, "scripts": { "build:web-view": "webpack --config ./webpack/webpack.config.web-view.ts", "build:main": "webpack --config ./webpack/webpack.config.main.ts", diff --git a/src/__tests__/interlinearizer.web-view.test.tsx b/src/__tests__/interlinearizer.web-view.test.tsx index f1a0d324..5306639e 100644 --- a/src/__tests__/interlinearizer.web-view.test.tsx +++ b/src/__tests__/interlinearizer.web-view.test.tsx @@ -4,18 +4,65 @@ import type { WebViewProps } from '@papi/core'; import type { SerializedVerseRef } from '@sillsdev/scripture'; -import { render, screen } from '@testing-library/react'; -import { InterlinearXmlParser } from 'parsers/interlinearXmlParser'; - -/** Mock parser to allow overriding constructor behavior per test. */ -jest.mock('parsers/interlinearXmlParser', () => { - const actual = jest.requireActual( - 'parsers/interlinearXmlParser', - ); - return { - InterlinearXmlParser: jest.fn().mockImplementation(() => new actual.InterlinearXmlParser()), - }; -}); +import { act, fireEvent, render, screen, waitFor } from '@testing-library/react'; +import * as lexiconParser from 'parsers/paratext-9/lexiconParser'; +import * as paratext9Parser from 'parsers/paratext-9/interlinearParser'; +import * as paratext9Converter from 'parsers/paratext-9/converter'; +import type { Analysis, InterlinearAlignment, Interlinearization } from 'interlinearizer'; +import { AnalysisType, Confidence } from 'types/interlinearizer-enums'; +import * as interlinearizerWebViewModule from '../interlinearizer.web-view'; + +jest.mock('parsers/paratext-9/interlinearParser'); +jest.mock('parsers/paratext-9/converter'); +jest.mock('parsers/paratext-9/lexiconParser'); + +type ParserMock = typeof paratext9Parser & { + mockParse: jest.Mock; + stubInterlinearData: import('parsers/paratext-9/types').InterlinearData; +}; + +/** Uses real converter types so stub properties (e.g. .books) are correctly typed in tests. */ +type ConverterMock = typeof paratext9Converter & { + mockConvertAlignment: jest.Mock; + mockCreateSourceAnalyses: jest.Mock; + mockCreateTargetAnalyses: jest.Mock; + stubAlignment: InterlinearAlignment; + stubInterlinearization: Interlinearization; + stubTargetInterlinearization: Interlinearization; + stubSourceAnalysesMap: Map; + stubTargetAnalysesMap: Map; + stubAnalysesMap: Map; +}; + +function isParserMock(m: typeof paratext9Parser): m is ParserMock { + return 'mockParse' in m && 'stubInterlinearData' in m; +} +function isConverterMock(m: typeof paratext9Converter): m is ConverterMock { + return 'mockConvertAlignment' in m && 'stubAlignment' in m; +} + +function getParserMock(): ParserMock { + if (!isParserMock(paratext9Parser)) throw new Error('Expected parser mock'); + return paratext9Parser; +} +function getConverterMock(): ConverterMock { + if (!isConverterMock(paratext9Converter)) throw new Error('Expected converter mock'); + return paratext9Converter; +} + +const { stubInterlinearData, mockParse } = getParserMock(); +const { + stubAlignment, + stubInterlinearization, + stubTargetInterlinearization, + stubTargetAnalysesMap, + mockConvertAlignment, + mockCreateSourceAnalyses, + mockCreateTargetAnalyses, +} = getConverterMock(); + +/** First book of stub target (for building alignment variants). */ +const stubTargetBook = stubTargetInterlinearization.books[0]; /** * Load the WebView module; it assigns the component to globalThis.webViewComponent. This pattern is @@ -24,9 +71,9 @@ jest.mock('parsers/interlinearXmlParser', () => { * component must require() the module and read globalThis. If the WebView export mechanism changes, * update this test accordingly. */ -require('../interlinearizer.web-view'); const InterlinearizerWebView = globalThis.webViewComponent; +const { handleJsonViewModeKeyDown } = interlinearizerWebViewModule; if (!InterlinearizerWebView) throw new Error('webViewComponent not loaded'); /** Minimal SerializedVerseRef for hook mock return. */ @@ -50,76 +97,1033 @@ const testWebViewProps: WebViewProps = { updateWebViewDefinition: () => true, }; +/** + * Renders the WebView and waits for the mount effect's async conversion to settle inside act(). The + * component calls convertParatext9ToInterlinearAlignment(parsed, lexiconData) in useEffect; when + * the promise resolves it calls setAlignment. Without waiting, that update runs after the test and + * triggers "An update to ... was not wrapped in act(...)". This helper flushes the async work so + * all state updates are wrapped. + */ +async function renderWebView(): Promise> { + return act(async () => { + const result = render(); + await Promise.resolve(); + await Promise.resolve(); + return result; + }); +} + describe('InterlinearizerWebView', () => { - it('renders the heading "Interlinearizer"', () => { - render(); + beforeEach(() => { + jest.clearAllMocks(); + }); + + it('renders the heading "Interlinearizer"', async () => { + await renderWebView(); expect(screen.getByRole('heading', { name: /interlinearizer/i })).toBeInTheDocument(); }); - it('renders the description mentioning test-data XML', () => { - render(); + it('renders the description mentioning test-data XML', async () => { + await renderWebView(); expect( screen.getByText(/raw json of the model parsed from/i, { exact: false }), ).toBeInTheDocument(); - expect(screen.getByText(/test-data\/Interlinear_en_MAT\.xml/i)).toBeInTheDocument(); + expect(screen.getByText(/test-data\/Interlinear_en_JHN\.xml/i)).toBeInTheDocument(); + }); + + it('renders the view mode switch (Interlinear / InterlinearData / Interlinearization / Analyses)', async () => { + await renderWebView(); + + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + expect(radiogroup).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlinear$/i })).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlineardata$/i })).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlinearization$/i })).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^analyses$/i })).toBeInTheDocument(); + expect(screen.getByText(/view json as:/i)).toBeInTheDocument(); }); - it('parses the bundled test XML and displays parsed JSON', () => { - render(); + it('displays InterlinearData JSON by default when parser returns data', async () => { + await renderWebView(); - expect(screen.getByText(/parsed interlinear data \(json\)/i)).toBeInTheDocument(); - expect(screen.getByText(/"GlossLanguage"/)).toBeInTheDocument(); - expect(screen.getByText(/"BookId"/)).toBeInTheDocument(); + expect(screen.getByText(/^InterlinearData \(JSON\):$/)).toBeInTheDocument(); + expect(screen.getByText(/glossLanguage/i)).toBeInTheDocument(); + expect(screen.getByText(/bookId/i)).toBeInTheDocument(); }); - it('displays parsed structure with expected verse data', () => { - render(); + it('displays parsed structure including glossLanguage and bookId values', async () => { + await renderWebView(); expect(screen.getByText(/"en"/)).toBeInTheDocument(); - expect(screen.getByText(/"MAT"/)).toBeInTheDocument(); + expect(screen.getByText(/"JHN"/)).toBeInTheDocument(); }); - it('does not show parse error when XML is valid', () => { - render(); + it('does not show parse error when parser succeeds', async () => { + await renderWebView(); expect(screen.queryByText(/^parse error$/i)).not.toBeInTheDocument(); }); - it('displays parse error when parser throws an Error (uses err.message)', () => { - const actual = jest.requireActual( - '../parsers/interlinearXmlParser', - ); - const realInstance = new actual.InterlinearXmlParser(); - const throwingParse = (): never => { + it('displays parse error when parser throws an Error (uses err.message)', async () => { + mockParse.mockImplementationOnce(() => { throw new Error('Invalid XML structure'); - }; - Object.defineProperty(realInstance, 'parse', { value: throwingParse, writable: true }); - jest.mocked(InterlinearXmlParser).mockImplementationOnce(() => realInstance); + }); - render(); + await renderWebView(); expect(screen.getByRole('heading', { name: /^parse error$/i })).toBeInTheDocument(); expect(screen.getByText(/invalid xml structure/i)).toBeInTheDocument(); }); - it('displays parse error when parser throws non-Error (uses String(err))', () => { - const actual = jest.requireActual( - '../parsers/interlinearXmlParser', - ); - const realInstance = new actual.InterlinearXmlParser(); - const throwingParse = (): never => { + it('switching to Interlinear shows rendered view with book ref', async () => { + await renderWebView(); + + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + expect(screen.getByText(/^Interlinear \(rendered\):$/)).toBeInTheDocument(); + expect(screen.getByRole('heading', { name: 'JHN', level: 2 })).toBeInTheDocument(); + }); + + it('Interlinear view shows segment ref, word surface text, gloss, and punctuation', async () => { + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN 1:1')).toBeInTheDocument(); + }); + expect(screen.getAllByText('In').length).toBeGreaterThanOrEqual(1); + expect(screen.getAllByText(',').length).toBeGreaterThanOrEqual(1); + }); + + it('Interlinear view shows "Converting Paratext 9 data…" while conversion is in flight', async () => { + let resolveConvert: ((value: typeof stubAlignment) => void) | undefined; + const convertPromise = new Promise((resolve) => { + resolveConvert = resolve; + }); + mockConvertAlignment.mockReturnValueOnce(convertPromise); + + await act(async () => { + render(); + await Promise.resolve(); + }); + + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + expect(screen.getByText(/Converting Paratext 9 data…/)).toBeInTheDocument(); + + await act(async () => { + if (resolveConvert) resolveConvert(stubAlignment); + await convertPromise; + }); + }); + + it('Interlinear view shows "No alignment or analyses available." when conversion rejects', async () => { + mockConvertAlignment.mockRejectedValueOnce(new Error('Conversion failed')); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText(/No alignment or analyses available\./)).toBeInTheDocument(); + }); + }); + + it('Interlinear view shows "No alignment or analyses available." when sourceAnalysesMap is undefined', async () => { + mockCreateSourceAnalyses.mockReturnValueOnce(undefined); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText(/No alignment or analyses available\./)).toBeInTheDocument(); + }); + }); + + it('Interlinear view shows surface form on Gloss row when word has assignment but analysis missing from map', async () => { + const interlinearWithMissingAnalysis = { + ...stubInterlinearization, + books: [ + { + id: 'mock-book-id', + bookRef: 'JHN', + textVersion: '', + segments: [ + { + id: 'seg-no-gloss', + segmentRef: 'JHN 1:2', + occurrences: [ + { + id: 'occ-no-analysis', + segmentId: 'seg-no-gloss', + index: 0, + anchor: '0-3', + surfaceText: 'the', + writingSystem: '', + type: 'word', + assignments: [ + { + id: 'assign-unknown', + occurrenceId: 'occ-no-analysis', + analysisId: 'nonexistent-analysis-id', + status: 'approved', + }, + ], + }, + ], + }, + ], + }, + ], + }; + const targetWithSameLayout = { + ...stubTargetInterlinearization, + books: [ + { + ...stubTargetBook, + segments: [ + { + id: 'seg-no-gloss-target', + segmentRef: 'JHN 1:2', + baselineText: '', + occurrences: [ + { + id: 'occ-no-analysis-target', + segmentId: 'seg-no-gloss-target', + index: 0, + anchor: '0-3', + surfaceText: 'the', + writingSystem: '', + type: 'word', + assignments: [], + }, + ], + }, + ], + }, + ], + }; + mockConvertAlignment.mockResolvedValueOnce({ + ...stubAlignment, + source: interlinearWithMissingAnalysis, + target: targetWithSameLayout, + }); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN 1:2')).toBeInTheDocument(); + }); + // Gloss row shows target occurrence.surfaceText, so "the" appears in both Source and Gloss rows + expect(screen.getAllByText('the').length).toBeGreaterThanOrEqual(1); + }); + + it('Interlinear view shows Analyses line with multiple glosses when word has multiple assignments', async () => { + const analysisId1 = 'target-analysis-multi-1'; + const analysisId2 = 'target-analysis-multi-2'; + const interlinearWithMultipleAnalyses = { + ...stubInterlinearization, + books: [ + { + id: 'mock-book-id', + bookRef: 'JHN', + textVersion: '', + segments: [ + { + id: 'seg-multi', + segmentRef: 'JHN 1:5', + occurrences: [ + { + id: 'occ-multi', + segmentId: 'seg-multi', + index: 0, + anchor: '0-4', + surfaceText: 'word', + writingSystem: '', + type: 'word', + assignments: [ + { + id: 'a1', + occurrenceId: 'occ-multi', + analysisId: 'analysis-multi-1', + status: 'approved', + }, + { + id: 'a2', + occurrenceId: 'occ-multi', + analysisId: 'analysis-multi-2', + status: 'approved', + }, + ], + }, + ], + }, + ], + }, + ], + }; + const targetSegmentMulti = { + id: 'seg-multi-target', + segmentRef: 'JHN 1:5', + baselineText: '', + occurrences: [ + { + id: 'occ-multi-target', + segmentId: 'seg-multi-target', + index: 0, + anchor: '0-4', + surfaceText: '', + writingSystem: '', + type: 'word', + assignments: [ + { + id: 'a1-t', + occurrenceId: 'occ-multi-target', + analysisId: analysisId1, + status: 'approved', + }, + { + id: 'a2-t', + occurrenceId: 'occ-multi-target', + analysisId: analysisId2, + status: 'approved', + }, + ], + }, + ], + }; + const targetAnalysesMapWithMultiple = new Map([ + ...stubTargetAnalysesMap, + [ + analysisId1, + { + id: analysisId1, + analysisLanguage: 'en', + analysisType: AnalysisType.Morph, + confidence: Confidence.Medium, + sourceSystem: 'paratext-9', + sourceUser: 'paratext-9-parser', + glossText: 'stem', + morphemeBundles: [ + { id: `${analysisId1}-bundle-0`, index: 0, form: 'stem', writingSystem: '' }, + ], + }, + ], + [ + analysisId2, + { + id: analysisId2, + analysisLanguage: 'en', + analysisType: AnalysisType.Morph, + confidence: Confidence.Medium, + sourceSystem: 'paratext-9', + sourceUser: 'paratext-9-parser', + glossText: 'suffix', + morphemeBundles: [ + { id: `${analysisId2}-bundle-0`, index: 0, form: '-suffix', writingSystem: '' }, + ], + }, + ], + ]); + mockConvertAlignment.mockResolvedValueOnce({ + ...stubAlignment, + source: interlinearWithMultipleAnalyses, + target: { + ...stubTargetInterlinearization, + books: [{ ...stubTargetBook, segments: [targetSegmentMulti] }], + }, + }); + mockCreateTargetAnalyses.mockReturnValueOnce(targetAnalysesMapWithMultiple); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN 1:5')).toBeInTheDocument(); + }); + expect(screen.getAllByText('word').length).toBeGreaterThanOrEqual(1); + expect(screen.getByText('stem suffix')).toBeInTheDocument(); + }); + + it('Interlinear view shows only morph glosses when occurrence has mixed morph and non-morph assignments', async () => { + const morphId = 'target-analysis-morph-only'; + const morphNullGlossId = 'target-analysis-morph-null-gloss'; + const wordformId = 'target-analysis-wordform'; + const sourceWithMixed = { + ...stubInterlinearization, + books: [ + { + id: 'mock-book-id', + bookRef: 'JHN', + textVersion: '', + segments: [ + { + id: 'seg-mixed', + segmentRef: 'JHN 1:6', + occurrences: [ + { + id: 'occ-mixed', + segmentId: 'seg-mixed', + index: 0, + anchor: '0-4', + surfaceText: 'word', + writingSystem: '', + type: 'word', + assignments: [ + { + id: 'a-morph', + occurrenceId: 'occ-mixed', + analysisId: morphId, + status: 'approved', + }, + { + id: 'a-morph-null', + occurrenceId: 'occ-mixed', + analysisId: morphNullGlossId, + status: 'approved', + }, + { + id: 'a-wf', + occurrenceId: 'occ-mixed', + analysisId: wordformId, + status: 'approved', + }, + ], + }, + ], + }, + ], + }, + ], + }; + const targetSegmentMixed = { + id: 'seg-mixed-target', + segmentRef: 'JHN 1:6', + baselineText: '', + occurrences: [ + { + id: 'occ-mixed-target', + segmentId: 'seg-mixed-target', + index: 0, + anchor: '0-4', + surfaceText: '', + writingSystem: '', + type: 'word', + assignments: [ + { + id: 'a-morph-t', + occurrenceId: 'occ-mixed-target', + analysisId: morphId, + status: 'approved', + }, + { + id: 'a-morph-null-t', + occurrenceId: 'occ-mixed-target', + analysisId: morphNullGlossId, + status: 'approved', + }, + { + id: 'a-wf-t', + occurrenceId: 'occ-mixed-target', + analysisId: wordformId, + status: 'approved', + }, + ], + }, + ], + }; + const targetAnalysesMapMixed = new Map([ + ...stubTargetAnalysesMap, + [ + morphId, + { + id: morphId, + analysisLanguage: 'en', + analysisType: AnalysisType.Morph, + confidence: Confidence.Medium, + sourceSystem: 'paratext-9', + sourceUser: 'paratext-9-parser', + glossText: 'onlyMorph', + morphemeBundles: [{ id: `${morphId}-bundle`, index: 0, form: 'only', writingSystem: '' }], + }, + ], + [ + morphNullGlossId, + { + id: morphNullGlossId, + analysisLanguage: 'en', + analysisType: AnalysisType.Morph, + confidence: Confidence.Medium, + sourceSystem: 'paratext-9', + sourceUser: 'paratext-9-parser', + glossText: undefined, + morphemeBundles: [ + { id: `${morphNullGlossId}-bundle`, index: 0, form: 'x', writingSystem: '' }, + ], + }, + ], + [ + wordformId, + { + id: wordformId, + analysisLanguage: 'en', + analysisType: AnalysisType.Wordform, + confidence: Confidence.Medium, + sourceSystem: 'paratext-9', + sourceUser: 'paratext-9-parser', + glossText: undefined, + morphemeBundles: [], + }, + ], + ]); + mockConvertAlignment.mockResolvedValueOnce({ + ...stubAlignment, + source: sourceWithMixed, + target: { + ...stubTargetInterlinearization, + books: [{ ...stubTargetBook, segments: [targetSegmentMixed] }], + }, + }); + mockCreateTargetAnalyses.mockReturnValueOnce(targetAnalysesMapMixed); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN 1:6')).toBeInTheDocument(); + }); + expect(screen.getByText('onlyMorph')).toBeInTheDocument(); + }); + + it('Interlinear view skips rendering book when target has fewer books than source', async () => { + const secondSourceBook = { + id: 'mock-book-2', + bookRef: 'MAT', + textVersion: '', + segments: [ + { + id: 'seg-mat-1', + segmentRef: 'MAT 1:1', + occurrences: [ + { + id: 'occ-mat', + segmentId: 'seg-mat-1', + index: 0, + anchor: '0-3', + surfaceText: 'One', + writingSystem: '', + type: 'word', + assignments: [], + }, + ], + }, + ], + }; + const sourceBooks = stubInterlinearization.books; + const sourceTwoBooks = { + ...stubInterlinearization, + books: [...sourceBooks, secondSourceBook], + }; + mockConvertAlignment.mockResolvedValueOnce({ + ...stubAlignment, + source: sourceTwoBooks, + target: stubTargetInterlinearization, + }); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN')).toBeInTheDocument(); + }); + expect(screen.getByText('JHN 1:1')).toBeInTheDocument(); + expect(screen.queryByText('MAT')).not.toBeInTheDocument(); + }); + + it('Interlinear view skips rendering segment when target book has fewer segments than source', async () => { + const secondSourceSegment = { + id: 'seg-source-2', + segmentRef: 'JHN 1:2', + occurrences: [ + { + id: 'occ-s2', + segmentId: 'seg-source-2', + index: 0, + anchor: '0-3', + surfaceText: 'the', + writingSystem: '', + type: 'word', + assignments: [], + }, + ], + }; + const firstBook = stubInterlinearization.books[0]; + const sourceTwoSegments = { + ...stubInterlinearization, + books: [ + { + ...firstBook, + segments: [...firstBook.segments, secondSourceSegment], + }, + ], + }; + mockConvertAlignment.mockResolvedValueOnce({ + ...stubAlignment, + source: sourceTwoSegments, + target: stubTargetInterlinearization, + }); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN 1:1')).toBeInTheDocument(); + }); + expect(screen.queryByText('JHN 1:2')).not.toBeInTheDocument(); + }); + + it('Interlinear view shows "—" on Gloss row for word with empty assignments', async () => { + const interlinearWithNoAssignments = { + ...stubInterlinearization, + books: [ + { + id: 'mock-book-id', + bookRef: 'JHN', + textVersion: '', + segments: [ + { + id: 'seg-empty-assign', + segmentRef: 'JHN 1:3', + occurrences: [ + { + id: 'occ-empty', + segmentId: 'seg-empty-assign', + index: 0, + anchor: '0-4', + surfaceText: 'word', + writingSystem: '', + type: 'word', + assignments: [], + }, + ], + }, + ], + }, + ], + }; + const targetNoAssignments = { + ...stubTargetInterlinearization, + books: [ + { + ...stubTargetBook, + segments: [ + { + id: 'seg-empty-assign-target', + segmentRef: 'JHN 1:3', + baselineText: '', + occurrences: [ + { + id: 'occ-empty-target', + segmentId: 'seg-empty-assign-target', + index: 0, + anchor: '0-4', + surfaceText: '', + writingSystem: '', + type: 'word', + assignments: [], + }, + ], + }, + ], + }, + ], + }; + mockConvertAlignment.mockResolvedValueOnce({ + ...stubAlignment, + source: interlinearWithNoAssignments, + target: targetNoAssignments, + }); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN 1:3')).toBeInTheDocument(); + }); + expect(screen.getAllByText('word').length).toBeGreaterThanOrEqual(1); + expect(screen.getByText('—')).toBeInTheDocument(); + }); + + it('Interlinear view shows placeholders when surfaceText is empty (· for word, — for punctuation)', async () => { + const interlinearWithEmptySurface = { + ...stubInterlinearization, + books: [ + { + id: 'mock-book-id', + bookRef: 'JHN', + textVersion: '', + segments: [ + { + id: 'seg-placeholders', + segmentRef: 'JHN 1:4', + occurrences: [ + { + id: 'occ-word', + segmentId: 'seg-placeholders', + index: 0, + anchor: '0-0', + surfaceText: '', + writingSystem: '', + type: 'word', + assignments: [], + }, + { + id: 'occ-punct', + segmentId: 'seg-placeholders', + index: 1, + anchor: '0-0', + surfaceText: '', + writingSystem: '', + type: 'punctuation', + assignments: [], + }, + ], + }, + ], + }, + ], + }; + const targetEmptySurface = { + ...stubTargetInterlinearization, + books: [ + { + ...stubTargetBook, + segments: [ + { + id: 'seg-placeholders-target', + segmentRef: 'JHN 1:4', + baselineText: '', + occurrences: [ + { + id: 'occ-word-t', + segmentId: 'seg-placeholders-target', + index: 0, + anchor: '0-0', + surfaceText: '', + writingSystem: '', + type: 'word', + assignments: [], + }, + { + id: 'occ-punct-t', + segmentId: 'seg-placeholders-target', + index: 1, + anchor: '0-0', + surfaceText: '', + writingSystem: '', + type: 'punctuation', + assignments: [], + }, + ], + }, + ], + }, + ], + }; + mockConvertAlignment.mockResolvedValueOnce({ + ...stubAlignment, + source: interlinearWithEmptySurface, + target: targetEmptySurface, + }); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + + await waitFor(() => { + expect(screen.getByText('JHN 1:4')).toBeInTheDocument(); + }); + expect(screen.getByText('·')).toBeInTheDocument(); + const emDashes = screen.getAllByText('—'); + expect(emDashes.length).toBeGreaterThanOrEqual(1); + }); + + it('switching to Interlinearization shows converted model JSON', async () => { + await renderWebView(); + + fireEvent.click(screen.getByRole('radio', { name: /^interlinearization$/i })); + + expect(screen.getByText(/^Interlinearization \(JSON\):$/)).toBeInTheDocument(); + await waitFor(() => expect(screen.getByText(/analysisLanguages/i)).toBeInTheDocument()); + await waitFor(() => expect(screen.getByText(/sourceWritingSystem/i)).toBeInTheDocument()); + await waitFor(() => expect(screen.getByText(/segments/i)).toBeInTheDocument()); + }); + + it('switching back to InterlinearData shows PT9 structure JSON', async () => { + await renderWebView(); + + fireEvent.click(screen.getByRole('radio', { name: /^interlinearization$/i })); + await waitFor(() => { + expect(screen.getByText(/^Interlinearization \(JSON\):$/)).toBeInTheDocument(); + }); + fireEvent.click(screen.getByRole('radio', { name: /^interlineardata$/i })); + + expect(screen.getByText(/^InterlinearData \(JSON\):$/)).toBeInTheDocument(); + expect(screen.getByText(/glossLanguage/i)).toBeInTheDocument(); + expect(screen.getByText(/bookId/i)).toBeInTheDocument(); + }); + + it('switching to Analyses shows analysis map JSON from test data', async () => { + await renderWebView(); + + fireEvent.click(screen.getByRole('radio', { name: /^analyses$/i })); + + expect(screen.getByText(/^Analyses \(JSON\):$/)).toBeInTheDocument(); + expect(mockCreateSourceAnalyses).toHaveBeenCalledWith(stubInterlinearData); + expect(mockCreateTargetAnalyses).toHaveBeenCalledWith(stubInterlinearData, expect.any(Object)); + expect(screen.getByText(/analysis-en-lex1-s1/)).toBeInTheDocument(); + expect(screen.getByText(/glossText/i)).toBeInTheDocument(); + expect(screen.getByText(/paratext-9/i)).toBeInTheDocument(); + }); + + it('Analyses view shows empty JSON pre when createSourceAnalyses returns undefined', async () => { + mockCreateSourceAnalyses.mockReturnValueOnce(undefined); + + const { container } = await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^analyses$/i })); + await waitFor(() => { + expect(screen.getByText(/^Analyses \(JSON\):$/)).toBeInTheDocument(); + }); + + const jsonPre = container.querySelector('pre'); + expect(jsonPre).toBeInTheDocument(); + expect(jsonPre).toBeEmptyDOMElement(); + expect(jsonPre).not.toHaveTextContent('undefined'); + }); + + it('uses no glossary when Lexicon parse throws (glossLookup undefined)', async () => { + jest.mocked(lexiconParser.parseLexicon).mockImplementationOnce(() => { + throw new Error('Invalid Lexicon XML'); + }); + + await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^analyses$/i })); + + await waitFor(() => { + expect(mockCreateTargetAnalyses).toHaveBeenCalledWith(stubInterlinearData, { + glossLookup: undefined, + }); + }); + }); + + it('renders empty JSON pre when jsonToShow is undefined (converter returns undefined)', async () => { + mockConvertAlignment.mockResolvedValueOnce(undefined); + + const { container } = await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinearization$/i })); + await waitFor(() => { + expect(container.querySelector('pre')).toBeInTheDocument(); + }); + + const jsonPre = container.querySelector('pre'); + expect(jsonPre).toBeInTheDocument(); + expect(jsonPre).toBeEmptyDOMElement(); + expect(jsonPre).not.toHaveTextContent('undefined'); + }); + + it('shows "Converting..." in Interlinearization view while conversion is in flight', async () => { + let resolveConvert: ((value: typeof stubAlignment) => void) | undefined; + const convertPromise = new Promise((resolve) => { + resolveConvert = resolve; + }); + mockConvertAlignment.mockReturnValueOnce(convertPromise); + + const { container } = await act(async () => { + const result = render(); + await Promise.resolve(); + return result; + }); + + fireEvent.click(screen.getByRole('radio', { name: /^interlinearization$/i })); + + await waitFor(() => { + const jsonPre = container.querySelector('pre'); + expect(jsonPre).toHaveTextContent('Converting...'); + }); + + await act(async () => { + if (resolveConvert) resolveConvert(stubAlignment); + await convertPromise; + }); + + await waitFor(() => { + expect(screen.getByText(/analysisLanguages/i)).toBeInTheDocument(); + }); + }); + + it('displays parse error when parser throws non-Error (uses String(err))', async () => { + mockParse.mockImplementationOnce(() => { // Intentionally throw a non-Error to test the String(err) branch in the catch block. // eslint-disable-next-line no-throw-literal -- testing non-Error handling throw 'plain string error'; - }; - Object.defineProperty(realInstance, 'parse', { value: throwingParse, writable: true }); - jest.mocked(InterlinearXmlParser).mockImplementationOnce(() => realInstance); + }); - render(); + await renderWebView(); expect(screen.getByRole('heading', { name: /^parse error$/i })).toBeInTheDocument(); expect(screen.getByText('plain string error')).toBeInTheDocument(); }); + + it('sets alignment to undefined when converter rejects', async () => { + mockConvertAlignment.mockRejectedValueOnce(new Error('Conversion failed')); + + const { container } = await renderWebView(); + fireEvent.click(screen.getByRole('radio', { name: /^interlinearization$/i })); + await waitFor(() => { + expect(container.querySelector('pre')).toBeInTheDocument(); + }); + + const jsonPre = container.querySelector('pre'); + expect(jsonPre).toBeInTheDocument(); + expect(jsonPre).toBeEmptyDOMElement(); + }); + + describe('handleJsonViewModeKeyDown', () => { + it('ArrowRight moves to next mode and updates selection', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + expect(screen.getByText(/^InterlinearData \(JSON\):$/)).toBeInTheDocument(); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowRight' }); + }); + + expect(screen.getByText(/^Interlinearization \(JSON\):$/)).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlinearization$/i })).toHaveAttribute( + 'aria-checked', + 'true', + ); + }); + + it('ArrowDown moves to next mode', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowDown' }); + }); + expect(screen.getByText(/^Interlinearization \(JSON\):$/)).toBeInTheDocument(); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowDown' }); + }); + expect(screen.getByText(/^Analyses \(JSON\):$/)).toBeInTheDocument(); + }); + + it('ArrowRight from last mode (Analyses) wraps to first (Interlinear)', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + fireEvent.click(screen.getByRole('radio', { name: /^analyses$/i })); + expect(screen.getByText(/^Analyses \(JSON\):$/)).toBeInTheDocument(); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowRight' }); + }); + + expect(screen.getByText(/^Interlinear \(rendered\):$/)).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlinear$/i })).toHaveAttribute( + 'aria-checked', + 'true', + ); + }); + + it('ArrowLeft moves to previous mode', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + fireEvent.click(screen.getByRole('radio', { name: /^analyses$/i })); + expect(screen.getByText(/^Analyses \(JSON\):$/)).toBeInTheDocument(); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowLeft' }); + }); + + expect(screen.getByText(/^Interlinearization \(JSON\):$/)).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlinearization$/i })).toHaveAttribute( + 'aria-checked', + 'true', + ); + }); + + it('ArrowUp moves to previous mode', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + fireEvent.click(screen.getByRole('radio', { name: /^interlinearization$/i })); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowUp' }); + }); + + expect(screen.getByText(/^InterlinearData \(JSON\):$/)).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlineardata$/i })).toHaveAttribute( + 'aria-checked', + 'true', + ); + }); + + it('ArrowLeft from first mode (Interlinear) wraps to last (Analyses)', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + fireEvent.click(screen.getByRole('radio', { name: /^interlinear$/i })); + expect(screen.getByText(/^Interlinear \(rendered\):$/)).toBeInTheDocument(); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowLeft' }); + }); + + expect(screen.getByText(/^Analyses \(JSON\):$/)).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^analyses$/i })).toHaveAttribute( + 'aria-checked', + 'true', + ); + }); + + it('non-arrow key does not change mode', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + expect(screen.getByText(/^InterlinearData \(JSON\):$/)).toBeInTheDocument(); + + fireEvent.keyDown(radiogroup, { key: 'a' }); + fireEvent.keyDown(radiogroup, { key: 'Enter' }); + expect(screen.getByText(/^InterlinearData \(JSON\):$/)).toBeInTheDocument(); + expect(screen.getByRole('radio', { name: /^interlineardata$/i })).toHaveAttribute( + 'aria-checked', + 'true', + ); + }); + + it('moves focus to the newly selected radio on arrow key', async () => { + await renderWebView(); + const radiogroup = screen.getByRole('radiogroup', { name: /view json as:/i }); + const interlinearizationRadio = screen.getByRole('radio', { + name: /^interlinearization$/i, + }); + + await act(async () => { + fireEvent.keyDown(radiogroup, { key: 'ArrowRight' }); + }); + + expect(document.activeElement).toBe(interlinearizationRadio); + }); + + it('does nothing when current view mode is not in JSON_VIEW_MODES (idx === -1)', () => { + const setJsonViewMode = jest.fn(); + const focusRadio = jest.fn(); + // Pass a value not in JSON_VIEW_MODES so findIndex returns -1; handler takes string for testability. + handleJsonViewModeKeyDown('invalid', 'ArrowRight', setJsonViewMode, focusRadio); + + expect(setJsonViewMode).not.toHaveBeenCalled(); + expect(focusRadio).not.toHaveBeenCalled(); + }); + }); }); diff --git a/src/__tests__/parsers/paratext-9/converter.test.ts b/src/__tests__/parsers/paratext-9/converter.test.ts new file mode 100644 index 00000000..973db1d4 --- /dev/null +++ b/src/__tests__/parsers/paratext-9/converter.test.ts @@ -0,0 +1,305 @@ +/** + * @file Unit tests for the Paratext 9 converter: convertParatext9ToInterlinearization and + * createAnalyses. These tests use the real converter (no mock) so conversion and + * segment/occurrence logic is covered. Covers: sort comparator (word before punctuation when same + * range), single-cluster branch (Word-only or WordParse-only per range), and dual-cluster path. + */ + +import fs from 'fs'; +import { + convertParatext9ToInterlinearization, + createAnalyses, + createTargetAnalyses, +} from 'parsers/paratext-9/converter'; +import { Paratext9Parser } from 'parsers/paratext-9/interlinearParser'; +import type { + InterlinearData, + VerseData, + ClusterData, + PunctuationData, +} from 'parsers/paratext-9/types'; +import { getTestDataPath } from '../../test-helpers'; + +/** Builds minimal VerseData for a single verse. */ +function buildVerseData(overrides: { + hash?: string; + clusters?: ClusterData[]; + punctuations?: PunctuationData[]; +}): VerseData { + return { + hash: '', + clusters: [], + punctuations: [], + ...overrides, + }; +} + +/** Builds minimal ClusterData. */ +function buildCluster( + textRange: { index: number; length: number }, + lexemeIds: string[], + senseIds: string[] = [], +): ClusterData { + const lexemes = lexemeIds.map((lexemeId, i) => ({ + lexemeId, + senseId: senseIds[i] ?? '', + })); + const lexemesId = lexemeIds.join('/'); + const id = `${lexemesId}/${textRange.index}-${textRange.length}`; + return { + textRange: { index: textRange.index, length: textRange.length }, + lexemes, + lexemesId, + id, + excluded: false, + }; +} + +/** Builds minimal PunctuationData. */ +function buildPunctuation( + textRange: { index: number; length: number }, + afterText: string, +): PunctuationData { + return { + textRange: { index: textRange.index, length: textRange.length }, + beforeText: '', + afterText, + }; +} + +describe('convertParatext9ToInterlinearization', () => { + it('converts parsed Interlinear XML to Interlinearization (single- and dual-cluster paths)', async () => { + const xml = fs.readFileSync(getTestDataPath('Interlinear_en_JHN.xml'), 'utf-8'); + const parser = new Paratext9Parser(); + const interlinearData = parser.parse(xml); + + const result = await convertParatext9ToInterlinearization(interlinearData); + + expect(result).toBeDefined(); + expect(result.analysisLanguages).toEqual(['en']); + expect(result.books).toHaveLength(1); + expect(result.books[0].bookRef).toBe('JHN'); + expect(result.books[0].segments.length).toBeGreaterThan(0); + + const firstSegment = result.books[0].segments[0]; + expect(firstSegment.segmentRef).toBe('JHN 1:1'); + expect(firstSegment.occurrences.length).toBeGreaterThan(0); + + const wordOccurrences = firstSegment.occurrences.filter((o) => o.type === 'word'); + const punctOccurrences = firstSegment.occurrences.filter((o) => o.type === 'punctuation'); + expect(wordOccurrences.length).toBeGreaterThan(0); + expect(punctOccurrences.length).toBeGreaterThan(0); + }); + + it('sorts word before punctuation when both share the same text range (sort comparator branch)', async () => { + const range = { index: 5, length: 1 }; + const wordOnlyCluster = buildCluster(range, ['Word:x'], ['sense1']); + const verseData = buildVerseData({ + clusters: [wordOnlyCluster], + punctuations: [buildPunctuation(range, '.')], + }); + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { 'TST 1:1': verseData }, + }; + + const result = await convertParatext9ToInterlinearization(interlinearData); + + const segment = result.books[0].segments[0]; + expect(segment.occurrences).toHaveLength(2); + expect(segment.occurrences[0].type).toBe('word'); + expect(segment.occurrences[1].type).toBe('punctuation'); + }); + + it('sort comparator returns 0 when two items have same range and same kind', async () => { + const range = { index: 3, length: 1 }; + const verseData = buildVerseData({ + clusters: [buildCluster({ index: 0, length: 2 }, ['Word:ab'], ['s1'])], + punctuations: [buildPunctuation(range, ','), buildPunctuation(range, ',')], + }); + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { 'TST 1:1': verseData }, + }; + + const result = await convertParatext9ToInterlinearization(interlinearData); + + const segment = result.books[0].segments[0]; + expect(segment.occurrences).toHaveLength(3); + }); + + it('computes book text version from verse hashes (computeBookTextVersion non-empty path)', async () => { + const verseData = buildVerseData({ + hash: 'abc123', + clusters: [buildCluster({ index: 0, length: 1 }, ['Word:x'], ['s1'])], + punctuations: [], + }); + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { 'TST 1:1': verseData }, + }; + const mockHasher = jest.fn().mockResolvedValue('mock-text-version'); + + const result = await convertParatext9ToInterlinearization(interlinearData, { + hashSha256Hex: mockHasher, + }); + + expect(mockHasher).toHaveBeenCalled(); + expect(result.books[0].textVersion).toBe('mock-text-version'); + }); + + it('uses single cluster for both assignments and surface when range has only Word (single-cluster branch)', async () => { + const verseData = buildVerseData({ + clusters: [buildCluster({ index: 0, length: 2 }, ['Word:In'], ['sense-in'])], + punctuations: [], + }); + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { 'TST 1:1': verseData }, + }; + + const result = await convertParatext9ToInterlinearization(interlinearData); + + const segment = result.books[0].segments[0]; + expect(segment.occurrences).toHaveLength(1); + expect(segment.occurrences[0].type).toBe('word'); + expect(segment.occurrences[0].surfaceText).toBe('In'); + expect(segment.occurrences[0].assignments).toHaveLength(0); + }); + + it('uses single cluster for both when range has only WordParse (single-cluster branch)', async () => { + const verseData = buildVerseData({ + clusters: [ + buildCluster( + { index: 0, length: 7 }, + ['Stem:run', 'Suffix:ning'], + ['sense-run', 'sense-ing'], + ), + ], + punctuations: [], + }); + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { 'TST 1:1': verseData }, + }; + + const result = await convertParatext9ToInterlinearization(interlinearData); + + const segment = result.books[0].segments[0]; + expect(segment.occurrences).toHaveLength(1); + expect(segment.occurrences[0].type).toBe('word'); + expect(segment.occurrences[0].surfaceText).toBe('running'); + expect(segment.occurrences[0].assignments).toHaveLength(2); + }); + + it('surfaceTextFromLexemes uses full lexemeId when no colon (clusterKind other path)', async () => { + const verseData = buildVerseData({ + clusters: [buildCluster({ index: 0, length: 3 }, ['noColonId'], ['s1'])], + punctuations: [], + }); + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { 'TST 1:1': verseData }, + }; + + const result = await convertParatext9ToInterlinearization(interlinearData); + + const segment = result.books[0].segments[0]; + expect(segment.occurrences[0].surfaceText).toBe('noColonId'); + }); +}); + +describe('createSourceAnalyses / createAnalyses', () => { + it('builds source analyses map with only WordParse lexemes (no Word analyses)', () => { + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { + 'TST 1:1': buildVerseData({ + clusters: [ + buildCluster( + { index: 0, length: 9 }, + ['Stem:begin', 'Suffix:ing'], + ['sense-stem', 'sense-suffix'], + ), + ], + punctuations: [], + }), + }, + }; + + const map = createAnalyses(interlinearData); + + expect(map.size).toBeGreaterThan(0); + const stemAnalysis = map.get('analysis-en-Stem:begin-sense-stem'); + const suffixAnalysis = map.get('analysis-en-Suffix:ing-sense-suffix'); + expect(stemAnalysis).toBeDefined(); + expect(suffixAnalysis).toBeDefined(); + expect(stemAnalysis?.analysisType).toBe('morph'); + }); + + it('createTargetAnalyses uses glossLookup when provided', () => { + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { + 'TST 1:1': buildVerseData({ + clusters: [ + buildCluster({ index: 0, length: 9 }, ['Stem:begin', 'Suffix:ing'], ['s1', 's2']), + ], + punctuations: [], + }), + }, + }; + const glossLookup: (senseId: string, _lang: string) => string | undefined = (senseId) => { + if (senseId === 's1') return 'hello'; + if (senseId === 's2') return 'world'; + return undefined; + }; + + const map = createTargetAnalyses(interlinearData, { glossLookup }); + + const analysis1 = map.get('target-analysis-en-Stem:begin-s1'); + const analysis2 = map.get('target-analysis-en-Suffix:ing-s2'); + expect(analysis1?.glossText).toBe('hello'); + expect(analysis2?.glossText).toBe('world'); + }); + + it('sets analysisType to morph and morphemeBundles for WordParse lexemes (Stem/Suffix/Prefix)', () => { + const interlinearData: InterlinearData = { + glossLanguage: 'en', + bookId: 'TST', + verses: { + 'TST 1:1': buildVerseData({ + clusters: [ + buildCluster( + { index: 0, length: 9 }, + ['Stem:begin', 'Suffix:ing'], + ['sense-stem', 'sense-suffix'], + ), + ], + punctuations: [], + }), + }, + }; + + const map = createAnalyses(interlinearData); + + const stemAnalysis = map.get('analysis-en-Stem:begin-sense-stem'); + const suffixAnalysis = map.get('analysis-en-Suffix:ing-sense-suffix'); + expect(stemAnalysis).toBeDefined(); + expect(suffixAnalysis).toBeDefined(); + expect(stemAnalysis?.analysisType).toBe('morph'); + expect(suffixAnalysis?.analysisType).toBe('morph'); + expect(stemAnalysis?.morphemeBundles).toHaveLength(1); + expect(stemAnalysis?.morphemeBundles?.[0].form).toBe('begin'); + expect(suffixAnalysis?.morphemeBundles).toHaveLength(1); + expect(suffixAnalysis?.morphemeBundles?.[0].form).toBe('-ing'); + }); +}); diff --git a/src/__tests__/parsers/interlinearXmlParser.test.ts b/src/__tests__/parsers/paratext-9/interlinearParser.test.ts similarity index 66% rename from src/__tests__/parsers/interlinearXmlParser.test.ts rename to src/__tests__/parsers/paratext-9/interlinearParser.test.ts index 875789f1..e4b3900e 100644 --- a/src/__tests__/parsers/interlinearXmlParser.test.ts +++ b/src/__tests__/parsers/paratext-9/interlinearParser.test.ts @@ -1,25 +1,25 @@ -/** @file Unit tests for {@link InterlinearXmlParser}. */ +/** @file Unit tests for {@link Paratext9Parser}. */ /// import * as fs from 'fs'; -import * as path from 'path'; -import { InterlinearXmlParser } from 'parsers/interlinearXmlParser'; +import { Paratext9Parser } from 'parsers/paratext-9/interlinearParser'; +import { getTestDataPath } from '../../test-helpers'; -describe('InterlinearXmlParser', () => { - let parser: InterlinearXmlParser; +describe('Paratext9Parser', () => { + let parser: Paratext9Parser; beforeEach(() => { - parser = new InterlinearXmlParser(); + parser = new Paratext9Parser(); }); describe('parse() - valid XML', () => { it('parses minimal valid XML with one verse and one cluster', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -33,30 +33,29 @@ describe('InterlinearXmlParser', () => { const result = parser.parse(xml); expect(result).toEqual({ - ScrTextName: '', - GlossLanguage: 'en', - BookId: 'MAT', - Verses: { - 'MAT 1:1': { - Hash: '', - Clusters: [ + glossLanguage: 'en', + bookId: 'JHN', + verses: { + 'JHN 1:1': { + hash: '', + clusters: [ { - TextRange: { Index: 0, Length: 4 }, - Lexemes: [{ LexemeId: 'Word:word', SenseId: 'sense1' }], - LexemesId: 'Word:word', - Id: 'Word:word/0-4', - Excluded: false, + textRange: { index: 0, length: 4 }, + lexemes: [{ lexemeId: 'Word:word', senseId: 'sense1' }], + lexemesId: 'Word:word', + id: 'Word:word/0-4', + excluded: false, }, ], - Punctuations: [], + punctuations: [], }, }, }); }); - it('parses optional ScrTextName and verse Hash', () => { + it('parses verse Hash', () => { const xml = ` - + RUT 3:1 @@ -72,13 +71,12 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.ScrTextName).toBe('MyProject'); - expect(result.Verses['RUT 3:1'].Hash).toBe('ABC123'); + expect(result.verses['RUT 3:1'].hash).toBe('ABC123'); }); it('parses purely numeric verse Hash', () => { const xml = ` - + RUT 3:1 @@ -94,16 +92,15 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.ScrTextName).toBe('MyProject'); - expect(result.Verses['RUT 3:1'].Hash).toBe('123456'); + expect(result.verses['RUT 3:1'].hash).toBe('123456'); }); it('parses cluster with multiple lexemes and builds LexemesId and Id correctly', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -117,21 +114,21 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - const cluster = result.Verses['MAT 1:1'].Clusters[0]; - expect(cluster.Lexemes).toEqual([ - { LexemeId: 'Stem:hello', SenseId: 'g1' }, - { LexemeId: 'Suffix:ing', SenseId: 'g2' }, + const cluster = result.verses['JHN 1:1'].clusters[0]; + expect(cluster.lexemes).toEqual([ + { lexemeId: 'Stem:hello', senseId: 'g1' }, + { lexemeId: 'Suffix:ing', senseId: 'g2' }, ]); - expect(cluster.LexemesId).toBe('Stem:hello/Suffix:ing'); - expect(cluster.Id).toBe('Stem:hello/Suffix:ing/5-5'); + expect(cluster.lexemesId).toBe('Stem:hello/Suffix:ing'); + expect(cluster.id).toBe('Stem:hello/Suffix:ing/5-5'); }); it('parses lexeme Id containing slash: LexemesId and Id preserve the slash (slash-safe)', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -144,18 +141,18 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - const cluster = result.Verses['MAT 1:1'].Clusters[0]; - expect(cluster.Lexemes).toEqual([{ LexemeId: 'Word:hello/world', SenseId: 'g1' }]); - expect(cluster.LexemesId).toBe('Word:hello/world'); - expect(cluster.Id).toBe('Word:hello/world/0-12'); + const cluster = result.verses['JHN 1:1'].clusters[0]; + expect(cluster.lexemes).toEqual([{ lexemeId: 'Word:hello/world', senseId: 'g1' }]); + expect(cluster.lexemesId).toBe('Word:hello/world'); + expect(cluster.id).toBe('Word:hello/world/0-12'); }); it('preserves slash when joining Lexeme Ids (multiple lexemes, one Id contains slash)', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -169,21 +166,21 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - const cluster = result.Verses['MAT 1:1'].Clusters[0]; - expect(cluster.Lexemes).toEqual([ - { LexemeId: 'Stem:foo/bar', SenseId: 'g1' }, - { LexemeId: 'Suffix:ing', SenseId: 'g2' }, + const cluster = result.verses['JHN 1:1'].clusters[0]; + expect(cluster.lexemes).toEqual([ + { lexemeId: 'Stem:foo/bar', senseId: 'g1' }, + { lexemeId: 'Suffix:ing', senseId: 'g2' }, ]); - expect(cluster.LexemesId).toBe('Stem:foo/bar/Suffix:ing'); - expect(cluster.Id).toBe('Stem:foo/bar/Suffix:ing/5-11'); + expect(cluster.lexemesId).toBe('Stem:foo/bar/Suffix:ing'); + expect(cluster.id).toBe('Stem:foo/bar/Suffix:ing/5-11'); }); it('parses cluster with no lexemes: Id is Index-Length only (no leading slash)', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -195,18 +192,18 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - const cluster = result.Verses['MAT 1:1'].Clusters[0]; - expect(cluster.Lexemes).toEqual([]); - expect(cluster.LexemesId).toBe(''); - expect(cluster.Id).toBe('10-3'); + const cluster = result.verses['JHN 1:1'].clusters[0]; + expect(cluster.lexemes).toEqual([]); + expect(cluster.lexemesId).toBe(''); + expect(cluster.id).toBe('10-3'); }); it('parses Lexeme without GlossId as empty SenseId', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -219,18 +216,18 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Clusters[0].Lexemes[0]).toEqual({ - LexemeId: 'Word:a', - SenseId: '', + expect(result.verses['JHN 1:1'].clusters[0].lexemes[0]).toEqual({ + lexemeId: 'Word:a', + senseId: '', }); }); it('parses Cluster with Excluded=true', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -244,15 +241,15 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Clusters[0].Excluded).toBe(true); + expect(result.verses['JHN 1:1'].clusters[0].excluded).toBe(true); }); it('parses Cluster with Excluded=false', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -266,15 +263,15 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Clusters[0].Excluded).toBe(false); + expect(result.verses['JHN 1:1'].clusters[0].excluded).toBe(false); }); it('parses Cluster without Excluded as Excluded=false', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -287,15 +284,15 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Clusters[0].Excluded).toBe(false); + expect(result.verses['JHN 1:1'].clusters[0].excluded).toBe(false); }); it('parses Punctuation with Range, BeforeText, AfterText', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -314,21 +311,21 @@ describe('InterlinearXmlParser', () => { const result = parser.parse(xml); // Parser uses trimValues: false, so tag text is not trimmed. - expect(result.Verses['MAT 1:1'].Punctuations).toEqual([ + expect(result.verses['JHN 1:1'].punctuations).toEqual([ { - TextRange: { Index: 34, Length: 2 }, - BeforeText: '? ', - AfterText: '? ', + textRange: { index: 34, length: 2 }, + beforeText: '? ', + afterText: '? ', }, ]); }); it('omits Punctuation entries without valid Range', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -350,20 +347,20 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Punctuations).toHaveLength(1); - expect(result.Verses['MAT 1:1'].Punctuations[0]).toEqual({ - TextRange: { Index: 1, Length: 2 }, - BeforeText: 'c', - AfterText: 'd', + expect(result.verses['JHN 1:1'].punctuations).toHaveLength(1); + expect(result.verses['JHN 1:1'].punctuations[0]).toEqual({ + textRange: { index: 1, length: 2 }, + beforeText: 'c', + afterText: 'd', }); }); it('omits Punctuation entries when Range Index or Length is not finite (missing or non-numeric)', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -396,20 +393,20 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Punctuations).toHaveLength(1); - expect(result.Verses['MAT 1:1'].Punctuations[0]).toEqual({ - TextRange: { Index: 5, Length: 1 }, - BeforeText: 'valid', - AfterText: '', + expect(result.verses['JHN 1:1'].punctuations).toHaveLength(1); + expect(result.verses['JHN 1:1'].punctuations[0]).toEqual({ + textRange: { index: 5, length: 1 }, + beforeText: 'valid', + afterText: '', }); }); it('parses Punctuation with valid Range but missing BeforeText/AfterText as empty strings', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -425,20 +422,20 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Punctuations).toHaveLength(1); - expect(result.Verses['MAT 1:1'].Punctuations[0]).toEqual({ - TextRange: { Index: 10, Length: 1 }, - BeforeText: '', - AfterText: '', + expect(result.verses['JHN 1:1'].punctuations).toHaveLength(1); + expect(result.verses['JHN 1:1'].punctuations[0]).toEqual({ + textRange: { index: 10, length: 1 }, + beforeText: '', + afterText: '', }); }); it('parses multiple verses and preserves verse keys', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -447,7 +444,7 @@ describe('InterlinearXmlParser', () => { - MAT 1:2 + JHN 1:2 @@ -460,36 +457,36 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(Object.keys(result.Verses)).toEqual(['MAT 1:1', 'MAT 1:2']); - expect(result.Verses['MAT 1:1'].Clusters[0].Lexemes[0].LexemeId).toBe('a'); - expect(result.Verses['MAT 1:2'].Clusters[0].Lexemes[0].LexemeId).toBe('b'); + expect(Object.keys(result.verses)).toEqual(['JHN 1:1', 'JHN 1:2']); + expect(result.verses['JHN 1:1'].clusters[0].lexemes[0].lexemeId).toBe('a'); + expect(result.verses['JHN 1:2'].clusters[0].lexemes[0].lexemeId).toBe('b'); }); it('parses item with missing VerseData as empty Hash, Clusters, Punctuations', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1']).toEqual({ - Hash: '', - Clusters: [], - Punctuations: [], + expect(result.verses['JHN 1:1']).toEqual({ + hash: '', + clusters: [], + punctuations: [], }); }); it('parses VerseData with no Cluster or Punctuation as empty arrays', () => { const xml = ` - + - MAT 1:11 + JHN 1:11 @@ -497,19 +494,19 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:11']).toEqual({ - Hash: '', - Clusters: [], - Punctuations: [], + expect(result.verses['JHN 1:11']).toEqual({ + hash: '', + clusters: [], + punctuations: [], }); }); it('parses VerseData with Punctuation but no Cluster (Cluster ?? [] branch)', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -523,18 +520,18 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses['MAT 1:1'].Clusters).toEqual([]); - expect(result.Verses['MAT 1:1'].Punctuations).toHaveLength(1); - expect(result.Verses['MAT 1:1'].Punctuations[0]).toEqual({ - TextRange: { Index: 0, Length: 1 }, - BeforeText: ',', - AfterText: ',', + expect(result.verses['JHN 1:1'].clusters).toEqual([]); + expect(result.verses['JHN 1:1'].punctuations).toHaveLength(1); + expect(result.verses['JHN 1:1'].punctuations[0]).toEqual({ + textRange: { index: 0, length: 1 }, + beforeText: ',', + afterText: ',', }); }); it('parses Verses with no item array (item ?? [] branch) as empty verses record', () => { const xml = ` - + @@ -542,14 +539,14 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(result.Verses).toEqual({}); - expect(result.GlossLanguage).toBe('en'); - expect(result.BookId).toBe('MAT'); + expect(result.verses).toEqual({}); + expect(result.glossLanguage).toBe('en'); + expect(result.bookId).toBe('JHN'); }); it('skips items with missing string (verse key)', () => { const xml = ` - + @@ -560,7 +557,7 @@ describe('InterlinearXmlParser', () => { - MAT 1:1 + JHN 1:1 @@ -573,50 +570,42 @@ describe('InterlinearXmlParser', () => { `; const result = parser.parse(xml); - expect(Object.keys(result.Verses)).toEqual(['MAT 1:1']); - expect(result.Verses['MAT 1:1'].Clusters[0].Lexemes[0].LexemeId).toBe('y'); + expect(Object.keys(result.verses)).toEqual(['JHN 1:1']); + expect(result.verses['JHN 1:1'].clusters[0].lexemes[0].lexemeId).toBe('y'); }); it('parses real test-data file without throwing', () => { - const xmlPath = path.join(__dirname, '..', '..', '..', 'test-data', 'Interlinear_en_MAT.xml'); + const xmlPath = getTestDataPath('Interlinear_en_JHN.xml'); const xml = fs.readFileSync(xmlPath, 'utf-8'); const result = parser.parse(xml); - expect(result.GlossLanguage).toBe('en'); - expect(result.BookId).toBe('MAT'); - expect(result.ScrTextName).toBe(''); - expect(Object.keys(result.Verses).length).toBeGreaterThan(0); - - const mat11 = result.Verses['MAT 1:1']; - expect(mat11).toBeDefined(); - expect(mat11.Hash).toBe('C8D38188'); - expect(mat11.Clusters.length).toBeGreaterThan(0); - const firstCluster = mat11.Clusters[0]; - expect(firstCluster.TextRange).toEqual({ Index: 5, Length: 5 }); - expect(firstCluster.Lexemes[0]).toEqual({ - LexemeId: 'Word:hello', - SenseId: 'WvbPwa9D', + expect(result.glossLanguage).toBe('en'); + expect(result.bookId).toBe('JHN'); + expect(Object.keys(result.verses).length).toBeGreaterThan(0); + + const jhn11 = result.verses['JHN 1:1']; + expect(jhn11).toBeDefined(); + expect(jhn11.hash).toBe(''); + expect(jhn11.clusters.length).toBeGreaterThan(0); + const firstCluster = jhn11.clusters[0]; + expect(firstCluster.textRange).toEqual({ index: 0, length: 2 }); + expect(firstCluster.lexemes[0]).toEqual({ + lexemeId: 'Word:In', + senseId: 'aef10f32', }); - expect(firstCluster.Id).toMatch(/^Word:hello\/5-5$/); + expect(firstCluster.id).toMatch(/^Word:In\/0-2$/); - const versesWithPunctuation = Object.values(result.Verses).filter( - (v) => v.Punctuations.length > 0, - ); - expect(versesWithPunctuation.length).toBeGreaterThan(0); - const [firstWithPunctuation] = versesWithPunctuation; - expect(firstWithPunctuation.Punctuations[0]).toHaveProperty('TextRange'); - expect(firstWithPunctuation.Punctuations[0]).toHaveProperty('BeforeText'); - expect(firstWithPunctuation.Punctuations[0]).toHaveProperty('AfterText'); + expect(Object.values(result.verses).every((v) => Array.isArray(v.punctuations))).toBe(true); }); }); describe('parse() - invalid XML / errors', () => { it('throws when root element is not InterlinearData', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -627,10 +616,10 @@ describe('InterlinearXmlParser', () => { it('throws when GlossLanguage is missing', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -646,7 +635,7 @@ describe('InterlinearXmlParser', () => { - MAT 1:1 + JHN 1:1 @@ -659,10 +648,10 @@ describe('InterlinearXmlParser', () => { it('throws when GlossLanguage is empty string', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -675,7 +664,7 @@ describe('InterlinearXmlParser', () => { it('throws when Verses element is missing', () => { const xml = ` - + `; expect(() => parser.parse(xml)).toThrow('Invalid XML: Missing Verses element'); @@ -683,10 +672,10 @@ describe('InterlinearXmlParser', () => { it('throws when Cluster is missing Range element', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -701,12 +690,12 @@ describe('InterlinearXmlParser', () => { ); }); - it('throws when Range is missing Index or Length', () => { + it('throws when Range is missing Index', () => { const xmlNoIndex = ` - + - MAT 1:1 + JHN 1:1 @@ -720,12 +709,14 @@ describe('InterlinearXmlParser', () => { expect(() => parser.parse(xmlNoIndex)).toThrow( 'Invalid XML: Range missing required Index or Length attributes', ); + }); + it('throws when Range is missing Length', () => { const xmlNoLength = ` - + - MAT 1:1 + JHN 1:1 @@ -743,10 +734,10 @@ describe('InterlinearXmlParser', () => { it('throws when Lexeme is missing Id attribute', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -762,10 +753,10 @@ describe('InterlinearXmlParser', () => { it('throws when the same verse reference appears in more than one item', () => { const xml = ` - + - MAT 1:1 + JHN 1:1 @@ -774,7 +765,7 @@ describe('InterlinearXmlParser', () => { - MAT 1:2 + JHN 1:2 @@ -783,7 +774,7 @@ describe('InterlinearXmlParser', () => { - MAT 1:1 + JHN 1:1 @@ -795,20 +786,20 @@ describe('InterlinearXmlParser', () => { `; expect(() => parser.parse(xml)).toThrow( - 'Invalid XML: Duplicate verse reference "MAT 1:1". At most one VerseData per reference is allowed.', + 'Invalid XML: Duplicate verse reference "JHN 1:1". At most one VerseData per reference is allowed.', ); }); }); describe('constructor and instance', () => { it('can be instantiated multiple times', () => { - const p1 = new InterlinearXmlParser(); - const p2 = new InterlinearXmlParser(); + const p1 = new Paratext9Parser(); + const p2 = new Paratext9Parser(); const xml = ` - + - MAT 1:1 + JHN 1:1 diff --git a/src/__tests__/parsers/paratext-9/lexiconParser.test.ts b/src/__tests__/parsers/paratext-9/lexiconParser.test.ts new file mode 100644 index 00000000..2f46ef8f --- /dev/null +++ b/src/__tests__/parsers/paratext-9/lexiconParser.test.ts @@ -0,0 +1,415 @@ +/** + * @file Unit tests for {@link parseLexiconAndBuildGlossLookup}, {@link toArray}, and + * {@link normalizeGloss}. + */ + +import { + parseLexiconAndBuildGlossLookup, + parseLexicon, + lexemeKeyId, + getWordLevelGlossForForm, + buildGlossLookupFromLexicon, + buildWordLevelGlossLookup, + toArray, + normalizeGloss, +} from 'parsers/paratext-9/lexiconParser'; +import fs from 'fs'; +import { getTestDataPath } from '../../test-helpers'; + +describe('normalizeGloss', () => { + it('returns default language and text when Gloss object has no @_Language (covers ?? branch)', () => { + const result = normalizeGloss({ '#text': 'gloss without lang key' }); + expect(result.lang).toBe('*'); + expect(result.text).toBe('gloss without lang key'); + }); + + it('returns language and text when Gloss object has @_Language', () => { + const result = normalizeGloss({ '@_Language': 'en', '#text': 'hello' }); + expect(result.lang).toBe('en'); + expect(result.text).toBe('hello'); + }); + + it('returns default language when Gloss is string', () => { + const result = normalizeGloss('plain string gloss'); + expect(result.lang).toBe('*'); + expect(result.text).toBe('plain string gloss'); + }); +}); + +describe('lexemeKeyId', () => { + it('returns "Type:LexicalForm" when homograph <= 1', () => { + expect(lexemeKeyId({ type: 'Word', lexicalForm: 'x', homograph: 1 })).toBe('Word:x'); + expect(lexemeKeyId({ type: 'Stem', lexicalForm: 'run', homograph: 0 })).toBe('Stem:run'); + }); + + it('returns "Type:LexicalForm:Homograph" when homograph > 1', () => { + expect(lexemeKeyId({ type: 'Word', lexicalForm: 'bank', homograph: 2 })).toBe('Word:bank:2'); + expect(lexemeKeyId({ type: 'Stem', lexicalForm: 'run', homograph: 3 })).toBe('Stem:run:3'); + }); +}); + +describe('toArray', () => { + it('returns empty array for undefined (branch: undefined)', () => { + expect(toArray(undefined)).toEqual([]); + }); + + it('returns same array when value is already an array (branch: array)', () => { + const arr = [{ id: 'a' }]; + expect(toArray(arr)).toBe(arr); + expect(toArray(arr)).toEqual([{ id: 'a' }]); + }); + + it('wraps single object in array when value is not an array (branch: single object)', () => { + const single = { id: 'single' }; + expect(toArray(single)).toEqual([single]); + }); +}); + +describe('parseLexiconAndBuildGlossLookup', () => { + it('throws when root element is not Lexicon', () => { + const xml = ''; + expect(() => parseLexiconAndBuildGlossLookup(xml)).toThrow( + 'Invalid XML: Missing Lexicon root element', + ); + }); + + it('returns lookup that yields undefined for empty Lexicon', () => { + const xml = ''; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('anyId', 'en')).toBeUndefined(); + }); + + it('returns lookup that yields undefined for senseId with no Gloss', () => { + const xml = ` + + + + + + + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('senseNoGloss', 'en')).toBeUndefined(); + }); + + it('returns gloss text for matching senseId and language', () => { + const xml = ` + + + + + + + + good + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('Fz1CNXo3', 'en')).toBe('good'); + }); + + it('returns empty string when Gloss element has no text for that language', () => { + const xml = ` + + + + + + + + + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('6wa5ZOr2', 'grc')).toBe(''); + }); + + it('returns undefined for wrong language when Sense has only other languages', () => { + const xml = ` + + + + + + + + בֹּקֶר + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('wq/iyJMV', 'hbo')).toBe('בֹּקֶר'); + expect(lookup('wq/iyJMV', 'en')).toBeUndefined(); + }); + + it('returns undefined for empty senseId', () => { + const xml = ''; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('', 'en')).toBeUndefined(); + }); + + it('adds no pairs for Sense with empty Id', () => { + const xml = ` + + + + + + + + ignored + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('', 'en')).toBeUndefined(); + expect(lookup('any', 'en')).toBeUndefined(); + }); + + it('adds no pairs for Sense with missing Id attribute', () => { + const xml = ` + + + + + + + + no id sense + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('any', 'en')).toBeUndefined(); + }); + + it('uses default language when Gloss has empty Language attribute', () => { + const xml = ` + + + + + + + + default gloss + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('noLangSense', '*')).toBe('default gloss'); + expect(lookup('noLangSense', 'en')).toBe('default gloss'); + }); + + it('uses default language when Gloss has no Language attribute (FXP returns string)', () => { + const xml = ` + + + + + + + + no lang attribute + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('omitLangSense', '*')).toBe('no lang attribute'); + expect(lookup('omitLangSense', 'en')).toBe('no lang attribute'); + }); + + it('uses default language when Gloss is object with missing Language attribute (covers ?? branch)', () => { + const xml = ` + + + + + + + + en only + no lang key + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('mixedLangSense', 'en')).toBe('en only'); + expect(lookup('mixedLangSense', '*')).toBe('no lang key'); + }); + + it('handles Entry with no Sense', () => { + const xml = ` + + + + + + + + +`; + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('any', 'en')).toBeUndefined(); + }); + + it('parses test-data/Lexicon.xml and resolves known sense IDs', () => { + const xmlPath = getTestDataPath('Lexicon.xml'); + const xml = fs.readFileSync(xmlPath, 'utf-8'); + const lookup = parseLexiconAndBuildGlossLookup(xml); + expect(lookup('aef10f32', 'en')).toBe('in'); + expect(lookup('69eeb5e0', 'en')).toBe('the'); + expect(lookup('69f3b5c7', 'en')).toBe('begin'); + }); +}); + +describe('parseLexicon and word-level gloss', () => { + it('parses Lexicon XML into LexiconData with entries keyed by lexeme Id', () => { + const xml = fs.readFileSync(getTestDataPath('Lexicon.xml'), 'utf-8'); + const lexicon = parseLexicon(xml); + expect(lexicon.language).toBe('en'); + expect(lexicon.entries['Word:beginning']).toBeDefined(); + expect(lexicon.entries['Word:beginning'].senses[0].id).toBe('a2598f23'); + expect(lexicon.entries['Stem:begin']).toBeDefined(); + expect(getWordLevelGlossForForm(lexicon, 'beginning', 'en')).toBe('beginning'); + expect(getWordLevelGlossForForm(lexicon, 'begin', 'en')).toBeUndefined(); + }); + + it('buildGlossLookupFromLexicon and buildWordLevelGlossLookup match parseLexiconAndBuildGlossLookup behaviour', () => { + const xml = fs.readFileSync(getTestDataPath('Lexicon.xml'), 'utf-8'); + const lexicon = parseLexicon(xml); + const glossLookup = buildGlossLookupFromLexicon(lexicon); + const wordLevelLookup = buildWordLevelGlossLookup(lexicon); + expect(glossLookup('aef10f32', 'en')).toBe('in'); + expect(wordLevelLookup('beginning', 'en')).toBe('beginning'); + }); + + it('getWordLevelGlossForForm returns fallback gloss when requested language has no exact match', () => { + const xml = ` + + + + + + + + default gloss text + + + + +`; + const lexicon = parseLexicon(xml); + expect(getWordLevelGlossForForm(lexicon, 'onlyDefault', 'fr')).toBe('default gloss text'); + expect(getWordLevelGlossForForm(lexicon, 'onlyDefault', 'en')).toBe('default gloss text'); + }); +}); + +describe('parseLexicon edge cases (branch coverage)', () => { + it('falls back to Word for unknown Lexeme Type and default type/homograph when attributes missing or invalid', () => { + const xml = ` + + + + + + + one + + + + + + oneb + + + + + + two + + + + + + three + + + + + + four + + + + + + five + + + +`; + const lexicon = parseLexicon(xml); + expect(lexicon.entries['Word:x']).toBeDefined(); + expect(lexicon.entries['Word:x'].key.type).toBe('Word'); + expect(lexicon.entries['Word:whitespaceType']).toBeDefined(); + expect(lexicon.entries['Word:whitespaceType'].key.type).toBe('Word'); + expect(lexicon.entries['Word::2']).toBeDefined(); + expect(lexicon.entries['Word::2'].key.lexicalForm).toBe(''); + expect(lexicon.entries['Word:y']).toBeDefined(); + expect(lexicon.entries['Word:y'].key.homograph).toBe(1); + expect(lexicon.entries['Word:noHomograph']).toBeDefined(); + expect(lexicon.entries['Word:noHomograph'].key.homograph).toBe(1); + expect(lexicon.entries['Word:noType']).toBeDefined(); + expect(lexicon.entries['Word:noType'].key.type).toBe('Word'); + }); + + it('uses default Language, FontName, and FontSize when root attributes missing or invalid', () => { + const xml = ` + + + +`; + const lexicon = parseLexicon(xml); + expect(lexicon.language).toBe(''); + expect(lexicon.fontName).toBe('Arial'); + expect(lexicon.fontSize).toBe(10); + }); + + it('uses default FontName when root FontName is whitespace-only and FontSize when invalid', () => { + const xml = ` + + + en + + nope + +`; + const lexicon = parseLexicon(xml); + expect(lexicon.language).toBe('en'); + expect(lexicon.fontName).toBe('Arial'); + expect(lexicon.fontSize).toBe(10); + }); +}); diff --git a/src/__tests__/test-helpers.ts b/src/__tests__/test-helpers.ts index 2669edc8..2394ff9d 100644 --- a/src/__tests__/test-helpers.ts +++ b/src/__tests__/test-helpers.ts @@ -1,10 +1,23 @@ /** * @file Test helpers used to build type-safe mocks without type assertions. Provides a minimal - * ExecutionActivationContext that satisfies @papi/core types. + * ExecutionActivationContext that satisfies @papi/core types, and a stable path resolver for the + * test-data directory. */ +import * as path from 'path'; + import type { ExecutionActivationContext } from '@papi/core'; import { UnsubscriberAsyncList } from 'platform-bible-utils'; +/** + * Resolves a path to a file under the project's test-data directory. + * + * @param relativePath - Filename or path relative to test-data (e.g. 'Interlinear_en_JHN.xml'). + * @returns Absolute path to the file under test-data. + */ +export function getTestDataPath(relativePath: string): string { + return path.resolve(__dirname, '..', '..', 'test-data', relativePath); +} + /** Minimal execution token-shaped object for tests (structural match for ExecutionToken). */ const mockExecutionToken: { type: 'extension'; diff --git a/src/interlinearizer.web-view.tsx b/src/interlinearizer.web-view.tsx index f47bd69a..f35b9ebb 100644 --- a/src/interlinearizer.web-view.tsx +++ b/src/interlinearizer.web-view.tsx @@ -1,22 +1,322 @@ -import { useMemo } from 'react'; -import type { InterlinearData } from 'interlinearizer'; -import { InterlinearXmlParser } from './parsers/interlinearXmlParser'; +import React, { useEffect, useMemo, useRef, useState } from 'react'; +import type { InterlinearData } from 'parsers/paratext-9/types'; +import { Paratext9Parser } from 'parsers/paratext-9/interlinearParser'; +import { + convertParatext9ToInterlinearAlignment, + createSourceAnalyses, + createTargetAnalyses, +} from 'parsers/paratext-9/converter'; +import { parseLexicon, buildGlossLookupFromLexicon } from 'parsers/paratext-9/lexiconParser'; -/** Test interlinear XML bundled at build time (from test-data/Interlinear_en_MAT.xml). */ -import testXml from '../test-data/Interlinear_en_MAT.xml?raw'; +import type { InterlinearAlignment, Segment, Occurrence, Analysis } from 'interlinearizer'; +import { OccurrenceType, AnalysisType } from 'types/interlinearizer-enums'; +/** Test interlinear XML bundled at build time (from test-data/Interlinear_en_JHN.xml). */ +import testXml from '../test-data/Interlinear_en_JHN.xml?raw'; +/** + * Lexicon XML for gloss text lookup (test-data/Lexicon.xml). Parsed once; on failure we use no + * glossary. + */ +import lexiconXml from '../test-data/Lexicon.xml?raw'; /** Result of parsing the bundled test XML: either data or an error message. */ type ParseResult = { data: InterlinearData; error: undefined } | { data: undefined; error: string }; +/** View mode: raw PT9, converted model, analyses map, or rendered interlinear. */ +export type JsonViewMode = 'interlinear-data' | 'interlinearization' | 'analyses' | 'interlinear'; + +/** + * Sentinel returned by jsonToShow when interlinearization mode is selected but conversion is still + * in progress. + */ +export const JSON_SHOW_CONVERTING = Symbol('JSON_SHOW_CONVERTING'); + +/** Ordered list of view modes for rendering and arrow-key navigation. */ +const JSON_VIEW_MODES: { key: JsonViewMode; label: string }[] = [ + { key: 'interlinear', label: 'Interlinear' }, + { key: 'interlinear-data', label: 'InterlinearData' }, + { key: 'interlinearization', label: 'Interlinearization' }, + { key: 'analyses', label: 'Analyses' }, +]; + +/** Returns the description for the view mode. */ +function getViewModeDescription(mode: JsonViewMode): string { + if (mode === 'interlinear') + return 'Rendered verse-by-verse: source morphs (above source), source text, gloss, and analyses (morpheme glosses below gloss when present).'; + if (mode === 'interlinear-data') return 'Paratext 9 book/verse/cluster structure.'; + if (mode === 'interlinearization') + return 'Converted interlinearizer book/segment/occurrence model.'; + return 'Analysis objects (ID → gloss, confidence, source) from test data.'; +} + +/** Returns the label for the view mode. */ +function getViewModeLabel(mode: JsonViewMode): string { + if (mode === 'interlinear') return 'Interlinear (rendered):'; + if (mode === 'interlinear-data') return 'InterlinearData (JSON):'; + if (mode === 'interlinearization') return 'Interlinearization (JSON):'; + return 'Analyses (JSON):'; +} + +/** Renders jsonToShow for the 
: "Converting…" for sentinel, stringified JSON, or empty string. */
+function formatJsonPreContent(jsonToShow: unknown): string {
+  if (jsonToShow === JSON_SHOW_CONVERTING) return 'Converting...';
+  if (jsonToShow !== undefined) return JSON.stringify(jsonToShow, undefined, 2);
+  return '';
+}
+
+/** Props for the rendered interlinear view (verse-by-verse source + target). */
+interface InterlinearDisplayProps {
+  /** Source and target interlinearizations (from convertParatext9ToInterlinearAlignment). */
+  alignment: InterlinearAlignment | undefined;
+  /** True while conversion is in progress. */
+  converting: boolean;
+  /** Source analysis ID → Analysis (morph forms for Source morphs row). */
+  sourceAnalysesMap: Map | undefined;
+  /** Target analysis ID → Analysis (gloss text for Analyses row). */
+  targetAnalysesMap: Map | undefined;
+}
+
+/**
+ * Morph-level glosses for an occurrence (for Analyses row). Only assignments whose analysis is
+ * Morph type; glossText from target analyses.
+ */
+function getMorphGlosses(occ: Occurrence, analysesMap: Map): string[] {
+  return occ.assignments
+    .map((a) => {
+      const analysis = analysesMap.get(a.analysisId);
+      return analysis?.analysisType === AnalysisType.Morph ? (analysis.glossText ?? '') : '';
+    })
+    .filter((t) => t.length > 0);
+}
+
+/**
+ * Resolves source morph forms for an occurrence (morpheme-level analyses only). Used for the
+ * "Source morphs" row above the source text, mirroring how target morph glosses appear below
+ * gloss.
+ *
+ * @param occ - Occurrence with assignments.
+ * @param analysesMap - Analysis ID → Analysis.
+ * @returns Ordered list of source forms (one per morph analysis; from morphemeBundles[].form).
+ */
+function getSourceMorphForms(occ: Occurrence, analysesMap: Map): string[] {
+  return occ.assignments
+    .map((a) => {
+      const analysis = analysesMap.get(a.analysisId);
+      if (analysis?.analysisType !== AnalysisType.Morph || !analysis.morphemeBundles?.length)
+        return undefined;
+      return analysis.morphemeBundles[0].form;
+    })
+    .filter((f): f is string => typeof f === 'string' && f.length > 0);
+}
+
+const LINE_LABEL_CLASS =
+  'tw-text-[10px] tw-uppercase tw-tracking-wide tw-text-muted-foreground/70 tw-shrink-0 tw-min-w-[5.5rem]';
+
+/**
+ * Renders a segment pair (source + target) as four labeled rows: Source morphs and Source from
+ * source interlinearization; Gloss and Analyses from target (occurrence.surfaceText = word-level
+ * gloss, morph analyses = Analyses row). One column per occurrence; labels align left.
+ */
+function SegmentBlock({
+  sourceSegment,
+  targetSegment,
+  sourceAnalysesMap,
+  targetAnalysesMap,
+}: {
+  sourceSegment: Segment;
+  targetSegment: Segment;
+  sourceAnalysesMap: Map;
+  targetAnalysesMap: Map;
+}) {
+  const n = sourceSegment.occurrences.length;
+  const gridCols = `minmax(5.5rem, max-content) repeat(${n}, minmax(0, max-content))`;
+  const cellClass = 'tw-text-left';
+  return (
+    

+      

+        {sourceSegment.segmentRef}
+      

+      

+        Source morphs
+        {sourceSegment.occurrences.map((occ) => {
+          const sourceMorphs = getSourceMorphForms(occ, sourceAnalysesMap);
+          const sourceMorphsLine =
+            occ.type === OccurrenceType.Word && sourceMorphs.length > 0
+              ? sourceMorphs.join(' ')
+              : '';
+          return (
+            
+              {sourceMorphsLine}
+            
+          );
+        })}
+        Source
+        {sourceSegment.occurrences.map((occ) => (
+          
+            {occ.type === OccurrenceType.Punctuation
+              ? occ.surfaceText || '—'
+              : occ.surfaceText || '·'}
+          
+        ))}
+        Gloss
+        {targetSegment.occurrences.map((occ) => (
+          
+            {occ.type === OccurrenceType.Punctuation
+              ? occ.surfaceText || ''
+              : occ.surfaceText || '—'}
+          
+        ))}
+        Analyses
+        {targetSegment.occurrences.map((occ) => {
+          const morphGlosses = getMorphGlosses(occ, targetAnalysesMap);
+          const analysesLine =
+            occ.type === OccurrenceType.Word && morphGlosses.length > 0
+              ? morphGlosses.join(' ')
+              : '';
+          return (
+            
+              {analysesLine}
+            
+          );
+        })}
+      

+    

+  );
+}
+
+/**
+ * Displays the interlinear alignment: one block per verse with source (morphs + surface) and target
+ * (gloss = occurrence.surfaceText, analyses = morph glosses).
+ */
+function InterlinearDisplay({
+  alignment,
+  converting,
+  sourceAnalysesMap,
+  targetAnalysesMap,
+}: InterlinearDisplayProps) {
+  if (converting) {
+    return 
Converting Paratext 9 data…
;
+  }
+  if (!alignment || !sourceAnalysesMap || !targetAnalysesMap) {
+    return (
+      
No alignment or analyses available.

+    );
+  }
+  const { source, target } = alignment;
+  return (
+    

+      {source.books.map((book, bookIdx) => {
+        const targetBook = target.books[bookIdx];
+        if (!targetBook) return undefined;
+        return (
+          

+            
{book.bookRef}

+            {book.segments.map((sourceSegment, segIdx) => {
+              const targetSegment = targetBook.segments[segIdx];
+              if (!targetSegment) return undefined;
+              return (
+                
+              );
+            })}
+          

+        );
+      })}
+    

+  );
+}
+
+/**
+ * Pure handler for arrow-key navigation on the JSON view mode radiogroup. Left/Up select previous,
+ * Right/Down select next. Exported for unit testing.
+ *
+ * @param currentMode - Current JSON view mode as string (must be in {@link JSON_VIEW_MODES} or
+ *   no-op).
+ * @param eventKey - KeyboardEvent.key (e.g. 'ArrowRight', 'ArrowLeft').
+ * @param setJsonViewMode - State setter for view mode.
+ * @param focusRadio - Callback to focus the radio for a given mode (e.g.
+ *   refs.current[key]?.focus()).
+ * @returns True if the key was handled (caller should call event.preventDefault()).
+ */
+export function handleJsonViewModeKeyDown(
+  currentMode: string,
+  eventKey: string,
+  setJsonViewMode: (mode: JsonViewMode) => void,
+  focusRadio: (mode: JsonViewMode) => void,
+): boolean {
+  const idx = JSON_VIEW_MODES.findIndex((m) => m.key === currentMode);
+  if (idx === -1) return false;
+  if (eventKey === 'ArrowRight' || eventKey === 'ArrowDown') {
+    const nextKey = JSON_VIEW_MODES[(idx + 1) % JSON_VIEW_MODES.length].key;
+    setJsonViewMode(nextKey);
+    focusRadio(nextKey);
+    return true;
+  }
+  if (eventKey === 'ArrowLeft' || eventKey === 'ArrowUp') {
+    const nextKey =
+      JSON_VIEW_MODES[(idx - 1 + JSON_VIEW_MODES.length) % JSON_VIEW_MODES.length].key;
+    setJsonViewMode(nextKey);
+    focusRadio(nextKey);
+    return true;
+  }
+  return false;
+}
+
 /**
  * Main interlinearizer WebView. Parses the bundled test XML into the interlinear model and displays
  * the result as raw JSON. No PAPI commands or file loading—everything is self-contained.
  *
- * Parser is created inside useMemo so parsing runs once per mount.
+ * A switch lets the user choose between: {@link InterlinearData} (Paratext 9 format),
+ * {@link Interlinearization} (converted interlinearizer model), or Analyses (ID → Analysis map
+ * derived from test data: gloss, confidence, source). Parser is created inside useMemo so parsing
+ * runs once per mount.
  */
 globalThis.webViewComponent = function InterlinearizerWebView() {
+  const [jsonViewMode, setJsonViewMode] = useState('interlinear-data');
+
+  /** Refs to each radio button for moving focus on arrow-key navigation. */
+  const radioRefs = useRef>({
+    interlinear: undefined,
+    'interlinear-data': undefined,
+    interlinearization: undefined,
+    analyses: undefined,
+  });
+
+  /** Wires arrow-key events to the pure handler and prevents default when handled. */
+  const onJsonViewModeKeyDown = (e: React.KeyboardEvent) => {
+    if (
+      handleJsonViewModeKeyDown(jsonViewMode, e.key, setJsonViewMode, (key) =>
+        radioRefs.current[key]?.focus(),
+      )
+    ) {
+      e.preventDefault();
+    }
+  };
+
   const { data: parsed, error: parseError } = useMemo((): ParseResult => {
-    const parser = new InterlinearXmlParser();
+    const parser = new Paratext9Parser();
     try {
       const data = parser.parse(testXml);
       return { data, error: undefined };
@@ -25,11 +325,90 @@ globalThis.webViewComponent = function InterlinearizerWebView() {
     }
   }, []);
 
+  const [alignment, setAlignment] = useState();
+  /**
+   * True once the convert promise has resolved or rejected; used to show "Converting…" only while
+   * in flight.
+   */
+  const [conversionSettled, setConversionSettled] = useState(false);
+
+  /** Parsed Lexicon (PT9-aligned). Built once; invalid Lexicon is ignored. */
+  const lexiconData = useMemo(() => {
+    try {
+      return parseLexicon(lexiconXml);
+    } catch {
+      return undefined;
+    }
+  }, []);
+
+  /** Gloss lookup (senseId, language) → text for target analyses. */
+  const glossLookup = useMemo(
+    () => (lexiconData ? buildGlossLookupFromLexicon(lexiconData) : undefined),
+    [lexiconData],
+  );
+
+  useEffect(() => {
+    if (!parsed) {
+      setAlignment(undefined);
+      setConversionSettled(false);
+      return;
+    }
+    setConversionSettled(false);
+    let cancelled = false;
+    convertParatext9ToInterlinearAlignment(parsed, lexiconData ?? undefined)
+      .then((result) => {
+        if (!cancelled) {
+          setAlignment(result);
+          setConversionSettled(true);
+        }
+        return undefined;
+      })
+      .catch(() => {
+        if (!cancelled) {
+          setAlignment(undefined);
+          setConversionSettled(true);
+        }
+      });
+    return () => {
+      cancelled = true;
+    };
+  }, [parsed, lexiconData]);
+
+  /** Source analyses (morph forms for Source morphs row). */
+  const sourceAnalysesMap = useMemo(
+    () => (parsed ? createSourceAnalyses(parsed) : undefined),
+    [parsed],
+  );
+  /** Target analyses (gloss text for Analyses row). */
+  const targetAnalysesMap = useMemo(
+    () => (parsed ? createTargetAnalyses(parsed, { glossLookup }) : undefined),
+    [parsed, glossLookup],
+  );
+
+  /**
+   * Data to show as JSON: depends on selected view mode. Shows converting sentinel when in
+   * interlinearization mode and conversion has not yet settled (promise still in flight).
+   */
+  const jsonToShow = useMemo((): unknown => {
+    if (jsonViewMode === 'interlinearization') {
+      if (alignment === undefined && !conversionSettled) return JSON_SHOW_CONVERTING;
+      return alignment;
+    }
+    if (jsonViewMode === 'analyses') {
+      if (!sourceAnalysesMap || !targetAnalysesMap) return undefined;
+      return {
+        source: Object.fromEntries(sourceAnalysesMap),
+        target: Object.fromEntries(targetAnalysesMap),
+      };
+    }
+    return parsed;
+  }, [jsonViewMode, parsed, alignment, conversionSettled, sourceAnalysesMap, targetAnalysesMap]);
+
   return (
     

       
Interlinearizer

       

-        Raw JSON of the model parsed from test-data/Interlinear_en_MAT.xml.
+        Raw JSON of the model parsed from test-data/Interlinear_en_JHN.xml.
       

 
       {parseError && (
@@ -43,10 +422,58 @@ globalThis.webViewComponent = function InterlinearizerWebView() {
 
       {parsed && (
         <>
-          
Parsed interlinear data (JSON):

-          
-            {JSON.stringify(parsed, undefined, 2)}
-          

+          

+            
+              View JSON as:
+            
+            

+              {JSON_VIEW_MODES.map(({ key, label }) => (
+                
+              ))}
+            

+            

+              {getViewModeDescription(jsonViewMode)}
+            

+          

+          
{getViewModeLabel(jsonViewMode)}

+          {jsonViewMode === 'interlinear' ? (
+            
+          ) : (
+            
+              {formatJsonPreContent(jsonToShow)}
+            

+          )}
         
       )}
     

diff --git a/src/parsers/paratext-9/__mocks__/converter.ts b/src/parsers/paratext-9/__mocks__/converter.ts
new file mode 100644
index 00000000..20a6ad27
--- /dev/null
+++ b/src/parsers/paratext-9/__mocks__/converter.ts
@@ -0,0 +1,181 @@
+/**
+ * @file Jest manual mock for parsers/paratext-9/converter. Placed adjacent to the module so
+ * jest.mock('parsers/paratext-9/converter') picks it up automatically. Used by
+ * interlinearizer.web-view tests so the WebView does not run real conversion.
+ */
+
+import type {
+  ConvertParatext9Options,
+  CreateAnalysesOptions,
+  CreateTargetAnalysesOptions,
+} from '../converter';
+import type { InterlinearData, LexiconData } from '../types';
+import type {
+  Analysis,
+  InterlinearAlignment,
+  Interlinearization,
+  Segment,
+} from 'interlinearizer';
+import { AnalysisType, AssignmentStatus, Confidence, OccurrenceType } from 'types/interlinearizer-enums';
+
+/** Stub source segment: one word occurrence (morph assignments) and one punctuation. */
+const stubSourceSegment: Segment = {
+  id: 'mock-seg-1',
+  segmentRef: 'JHN 1:1',
+  occurrences: [
+    {
+      id: 'mock-occ-word',
+      segmentId: 'mock-seg-1',
+      index: 0,
+      anchor: '0-2',
+      surfaceText: 'In',
+      writingSystem: '',
+      type: OccurrenceType.Word,
+      assignments: [
+        {
+          id: 'mock-assign-1',
+          occurrenceId: 'mock-occ-word',
+          analysisId: 'analysis-en-lex1-s1',
+          status: AssignmentStatus.Approved,
+        },
+      ],
+    },
+    {
+      id: 'mock-occ-punct',
+      segmentId: 'mock-seg-1',
+      index: 1,
+      anchor: '2-3',
+      surfaceText: ',',
+      writingSystem: '',
+      type: OccurrenceType.Punctuation,
+      assignments: [],
+    },
+  ],
+};
+
+/** Stub target segment: same layout; word occurrence has surfaceText = word-level gloss. */
+const stubTargetSegment: Segment = {
+  id: 'mock-seg-1-target',
+  segmentRef: 'JHN 1:1',
+  occurrences: [
+    {
+      id: 'mock-occ-word-target',
+      segmentId: 'mock-seg-1-target',
+      index: 0,
+      anchor: '0-2',
+      surfaceText: 'In',
+      writingSystem: '',
+      type: OccurrenceType.Word,
+      assignments: [
+        {
+          id: 'mock-assign-1-target',
+          occurrenceId: 'mock-occ-word-target',
+          analysisId: 'target-analysis-en-lex1-s1',
+          status: AssignmentStatus.Approved,
+        },
+      ],
+    },
+    {
+      id: 'mock-occ-punct-target',
+      segmentId: 'mock-seg-1-target',
+      index: 1,
+      anchor: '2-3',
+      surfaceText: ',',
+      writingSystem: '',
+      type: OccurrenceType.Punctuation,
+      assignments: [],
+    },
+  ],
+};
+
+/** Stub source Interlinearization (source side of alignment). */
+export const stubInterlinearization: Interlinearization = {
+  id: 'mock-interlinear-id',
+  sourceWritingSystem: '',
+  analysisLanguages: ['en'],
+  books: [
+    { id: 'mock-book-id', bookRef: 'JHN', textVersion: '', segments: [stubSourceSegment] },
+  ],
+};
+
+/** Stub target Interlinearization (target side; gloss in occurrence.surfaceText). */
+export const stubTargetInterlinearization: Interlinearization = {
+  id: 'mock-interlinear-id-target',
+  sourceWritingSystem: '',
+  analysisLanguages: ['en'],
+  books: [
+    {
+      id: 'mock-book-id-target',
+      bookRef: 'JHN',
+      textVersion: '',
+      segments: [stubTargetSegment],
+    },
+  ],
+};
+
+/** Stub InterlinearAlignment returned by mockConvertAlignment. */
+export const stubAlignment: InterlinearAlignment = {
+  id: 'mock-interlinear-id-alignment',
+  source: stubInterlinearization,
+  target: stubTargetInterlinearization,
+  links: [],
+};
+
+/** Source analyses (morph form for Source morphs row). */
+export const stubSourceAnalysesMap: Map = new Map([
+  [
+    'analysis-en-lex1-s1',
+    {
+      id: 'analysis-en-lex1-s1',
+      analysisLanguage: 'en',
+      analysisType: AnalysisType.Morph,
+      confidence: Confidence.Medium,
+      sourceSystem: 'paratext-9',
+      sourceUser: 'paratext-9-parser',
+      morphemeBundles: [{ id: 'bundle-0', index: 0, form: 'In', writingSystem: '' }],
+    },
+  ],
+]);
+
+/** Target analyses (gloss text for Analyses row). */
+export const stubTargetAnalysesMap: Map = new Map([
+  [
+    'target-analysis-en-lex1-s1',
+    {
+      id: 'target-analysis-en-lex1-s1',
+      analysisLanguage: 'en',
+      analysisType: AnalysisType.Morph,
+      confidence: Confidence.Medium,
+      sourceSystem: 'paratext-9',
+      sourceUser: 'paratext-9-parser',
+      glossText: 'sense1',
+      morphemeBundles: [{ id: 'bundle-0', index: 0, form: 'In', writingSystem: '' }],
+    },
+  ],
+]);
+
+/** Legacy: single analyses map for tests that still expect createAnalyses. */
+export const stubAnalysesMap = stubSourceAnalysesMap;
+
+/** Typed mocks so tests get correct argument/return types when calling the replaced converter. */
+export const mockConvert = jest
+  .fn, [InterlinearData, ConvertParatext9Options?]>()
+  .mockResolvedValue(stubInterlinearization);
+export const mockConvertAlignment = jest
+  .fn, [InterlinearData, LexiconData | undefined, ConvertParatext9Options?]>()
+  .mockResolvedValue(stubAlignment);
+export const mockCreateAnalyses = jest
+  .fn, [InterlinearData, CreateAnalysesOptions?]>()
+  .mockReturnValue(stubSourceAnalysesMap);
+export const mockCreateSourceAnalyses = jest
+  .fn, [InterlinearData]>()
+  .mockReturnValue(stubSourceAnalysesMap);
+export const mockCreateTargetAnalyses = jest
+  .fn, [InterlinearData, CreateTargetAnalysesOptions?]>()
+  .mockReturnValue(stubTargetAnalysesMap);
+
+export const convertParatext9ToInterlinearization = mockConvert;
+export const convertParatext9ToInterlinearAlignment = mockConvertAlignment;
+export const createAnalyses = mockCreateAnalyses;
+export const createSourceAnalyses = mockCreateSourceAnalyses;
+export const createTargetAnalyses = mockCreateTargetAnalyses;
diff --git a/src/parsers/paratext-9/__mocks__/interlinearParser.ts b/src/parsers/paratext-9/__mocks__/interlinearParser.ts
new file mode 100644
index 00000000..267d62bc
--- /dev/null
+++ b/src/parsers/paratext-9/__mocks__/interlinearParser.ts
@@ -0,0 +1,20 @@
+/**
+ * @file Jest manual mock for parsers/paratext-9/interlinearParser. Placed adjacent to the module so
+ * jest.mock('parsers/paratext-9/interlinearParser') picks it up automatically. Used by
+ * interlinearizer.web-view tests so the WebView does not run real XML parsing.
+ */
+
+import type { InterlinearData } from '../types';
+
+/** Stub InterlinearData returned by mockParse. Matches shape the WebView displays. */
+export const stubInterlinearData: InterlinearData = {
+  glossLanguage: 'en',
+  bookId: 'JHN',
+  verses: {},
+};
+
+export const mockParse = jest.fn().mockReturnValue(stubInterlinearData);
+
+export const Paratext9Parser = jest.fn().mockImplementation(() => ({
+  parse: mockParse,
+}));
diff --git a/src/parsers/paratext-9/__mocks__/lexiconParser.ts b/src/parsers/paratext-9/__mocks__/lexiconParser.ts
new file mode 100644
index 00000000..245ee051
--- /dev/null
+++ b/src/parsers/paratext-9/__mocks__/lexiconParser.ts
@@ -0,0 +1,17 @@
+/**
+ * @file Jest manual mock for parsers/paratext-9/lexiconParser. Placed adjacent to the module so
+ * jest.mock('parsers/paratext-9/lexiconParser') picks it up automatically. Used by
+ * interlinearizer.web-view tests so the WebView does not run real conversion.
+ */
+
+import { LexiconData } from "../types";
+
+/** Stub lookup: (senseId, language) => undefined. Matches LexiconGlossLookup shape. */
+const stubGlossLookup = (_senseId: string, _language: string): string | undefined => undefined;
+
+/** Minimal LexiconData so the webview can call buildGlossLookupFromLexicon(lexiconData). */
+const stubLexiconData: LexiconData = { language: 'en', fontName: '', fontSize: 0, entries: {} };
+
+export const parseLexicon = jest.fn().mockReturnValue(stubLexiconData);
+export const buildGlossLookupFromLexicon = jest.fn().mockReturnValue(stubGlossLookup);
+export const parseLexiconAndBuildGlossLookup = jest.fn().mockReturnValue(stubGlossLookup);
diff --git a/src/parsers/paratext-9/converter.ts b/src/parsers/paratext-9/converter.ts
new file mode 100644
index 00000000..d84166cb
--- /dev/null
+++ b/src/parsers/paratext-9/converter.ts
@@ -0,0 +1,830 @@
+/**
+ * @file Converts Paratext 9 interlinear data structures to the interlinearizer model.
+ *
+ *   This module converts from {@link InterlinearData} (types) to {@link Interlinearization}
+ *   (interlinearizer types). Interlinear XML is the source side (words/wordParses and ranges). The
+ *   Lexicon is the target side (gloss-language senses and gloss text). Mapping:
+ *   verse/cluster/source-lexeme + GlossId → book/segment/occurrence/analysis, with glossText from
+ *   Lexicon lookup.
+ */
+
+import type {
+  Interlinearization,
+  InterlinearAlignment,
+  AnalyzedBook,
+  Segment,
+  Analysis,
+  Occurrence,
+  AnalysisAssignment,
+} from 'interlinearizer';
+import {
+  OccurrenceType,
+  AnalysisType,
+  AssignmentStatus,
+  Confidence,
+} from 'types/interlinearizer-enums';
+import type {
+  InterlinearData,
+  VerseData,
+  StringRange,
+  ClusterData,
+  PunctuationData,
+  LexiconData,
+} from './types';
+import type { LexiconGlossLookup } from './lexiconParser';
+import { buildGlossLookupFromLexicon, getWordLevelGlossForForm } from './lexiconParser';
+
+/**
+ * Default SHA-256 hex implementation using the Web Crypto API so the converter can run in WebViews.
+ *
+ * @param input - UTF-8 string to hash.
+ * @returns Promise that resolves to the hex-encoded SHA-256 digest.
+ */
+/* c8 ignore start -- Web Crypto path; exercised in browser/WebView, not in Jest/jsdom */
+async function sha256HexWebCrypto(input: string): Promise {
+  const encoder = new TextEncoder();
+  const data = encoder.encode(input);
+  const hashBuffer = await globalThis.crypto.subtle.digest('SHA-256', data);
+  const hashArray = Array.from(new Uint8Array(hashBuffer));
+  return hashArray.map((b) => b.toString(16).padStart(2, '0')).join('');
+}
+/* c8 ignore end */
+
+/**
+ * Computes a stable book-level text version from verse hashes.
+ *
+ * Collects all non-empty verse hashes, sorts them deterministically, concatenates them, and returns
+ * the SHA-256 digest in hex. Used so that textVersion reflects changes in any verse. Uses the
+ * provided hasher or the default Web Crypto implementation (for WebViews).
+ *
+ * @param verseDataArray - Verse data in deterministic (e.g. sorted by ref) order.
+ * @param hashSha256Hex - Optional hasher; when omitted, uses Web Crypto. In Node contexts pass one
+ *   that matches paranext-core's generateHashFromBuffer('sha256', 'hex', Buffer.from(str,
+ *   'utf8')).
+ * @returns Promise that resolves to the hex SHA-256 digest, or '' if no verse hashes.
+ */
+async function computeBookTextVersion(
+  verseDataArray: VerseData[],
+  hashSha256Hex: (input: string) => Promise,
+): Promise {
+  const nonEmptyHashes = verseDataArray
+    .map((vd) => vd.hash)
+    .filter((h): h is string => h.length > 0);
+  if (nonEmptyHashes.length === 0) return '';
+  const sortedHashes = [...nonEmptyHashes].sort();
+  const concatenated = sortedHashes.join('');
+  return hashSha256Hex(concatenated);
+}
+
+/**
+ * Generates a deterministic ID for an interlinearization from Paratext 9 data.
+ *
+ * @param bookId - Book ID from InterlinearData.
+ * @returns A unique ID for the interlinearization.
+ */
+function generateInterlinearizationId(bookId: string): string {
+  return `${bookId}-interlinear`.toLowerCase().replace(/\s+/g, '-');
+}
+
+/**
+ * Generates a deterministic ID for an analyzed book.
+ *
+ * @param bookId - Book ID.
+ * @returns A unique ID for the book.
+ */
+function generateBookId(bookId: string): string {
+  return bookId.toLowerCase().replace(/\s+/g, '-');
+}
+
+/**
+ * Generates a deterministic ID for a segment (verse).
+ *
+ * @param verseRef - Verse reference (e.g., "MAT 1:1").
+ * @returns A unique ID for the segment.
+ */
+function generateSegmentId(verseRef: string): string {
+  return verseRef.toLowerCase().replace(/\s+/g, '-');
+}
+
+/**
+ * Generates a deterministic ID for an occurrence from a cluster.
+ *
+ * @param segmentId - Parent segment ID.
+ * @param clusterId - Cluster ID from ClusterData.
+ * @param index - Zero-based index within the segment.
+ * @returns A unique ID for the occurrence.
+ */
+function generateOccurrenceIdFromCluster(
+  segmentId: string,
+  clusterId: string,
+  index: number,
+): string {
+  return `${segmentId}-occ-${index}-${clusterId}`;
+}
+
+/**
+ * Generates a deterministic ID for an occurrence from punctuation.
+ *
+ * @param segmentId - Parent segment ID.
+ * @param textRange - Text range of the punctuation.
+ * @param index - Zero-based index within the segment.
+ * @returns A unique ID for the occurrence.
+ */
+function generateOccurrenceIdFromPunctuation(
+  segmentId: string,
+  textRange: StringRange,
+  index: number,
+): string {
+  return `${segmentId}-punc-${index}-${textRange.index}-${textRange.length}`;
+}
+
+/**
+ * Generates a deterministic ID for an analysis from lexeme data.
+ *
+ * @param lexemeId - Source word/wordParse ID from Interlinear XML (e.g. "Word:In", "Stem:begin").
+ * @param senseId - Target sense ID (GlossId); references Lexicon Sense for gloss text.
+ * @param glossLanguage - Gloss language code.
+ * @returns A unique ID for the analysis.
+ */
+function generateAnalysisId(lexemeId: string, senseId: string, glossLanguage: string): string {
+  const sensePart = senseId ? `-${senseId}` : '';
+  return `analysis-${glossLanguage}-${lexemeId}${sensePart}`;
+}
+
+/**
+ * Generates a deterministic ID for an analysis assignment.
+ *
+ * @param occurrenceId - Occurrence ID.
+ * @param analysisId - Analysis ID.
+ * @returns A unique ID for the assignment.
+ */
+function generateAssignmentId(occurrenceId: string, analysisId: string): string {
+  return `assign-${occurrenceId}-${analysisId}`;
+}
+
+/** Prefix for target-side analysis IDs so they do not collide with source-side. */
+const TARGET_ANALYSIS_ID_PREFIX = 'target-';
+
+/**
+ * Converts a text range to an anchor string.
+ *
+ * @param textRange - Character range in source text.
+ * @returns Anchor string in format "index-length".
+ */
+function textRangeToAnchor(textRange: StringRange): string {
+  return `${textRange.index}-${textRange.length}`;
+}
+
+/** Paratext 9 cluster kind: Word (single Word lexeme), WordParse (Stem/Suffix/Prefix), or other. */
+function clusterKind(cluster: ClusterData): 'word' | 'wordParse' | 'other' {
+  if (cluster.lexemes.length === 0) return 'other';
+  const types = new Set(
+    cluster.lexemes.map((l) =>
+      l.lexemeId.indexOf(':') >= 0 ? l.lexemeId.slice(0, l.lexemeId.indexOf(':')) : '',
+    ),
+  );
+  if (cluster.lexemes.length === 1 && (types.has('Word') || (types.size === 1 && types.has(''))))
+    return 'word';
+  if ([...types].some((t) => t === 'Stem' || t === 'Suffix' || t === 'Prefix')) return 'wordParse';
+  return 'other';
+}
+
+/** Whether the lexeme ID is a WordParse type (Stem, Suffix, or Prefix). */
+function isWordParseLexemeId(lexemeId: string): boolean {
+  const prefix = lexemeId.indexOf(':') >= 0 ? lexemeId.slice(0, lexemeId.indexOf(':')) : '';
+  return prefix === 'Stem' || prefix === 'Suffix' || prefix === 'Prefix';
+}
+
+/** Source morpheme form from a lexeme ID (part after first colon; e.g. "Stem:begin" → "begin"). */
+function sourceFormFromLexemeId(lexemeId: string): string {
+  const colon = lexemeId.indexOf(':');
+  return colon >= 0 ? lexemeId.slice(colon + 1) : lexemeId;
+}
+
+/**
+ * Display form for source morphs: affixes get hyphens (Suffix → leading "-", Prefix → trailing
+ * "-").
+ */
+function sourceMorphDisplayForm(lexemeId: string): string {
+  const form = sourceFormFromLexemeId(lexemeId);
+  if (lexemeId.startsWith('Suffix:')) return form.startsWith('-') ? form : `-${form}`;
+  if (lexemeId.startsWith('Prefix:')) return form.endsWith('-') ? form : `${form}-`;
+  return form;
+}
+
+/**
+ * Derives a display form for a cluster from its source-word lexeme IDs when surface text is not
+ * provided (e.g. Paratext 9 Interlinear XML only has Range + Lexeme Id/GlossId). Lexeme IDs are
+ * source-side (e.g. "Word:form", "Stem:form", "Suffix:form"); the part after the first colon is
+ * used. Multiple lexemes are concatenated (e.g. Stem:hello + Suffix:ing → "helloing").
+ *
+ * @param cluster - Cluster with at least one lexeme (source text words/wordParses).
+ * @returns Non-empty string suitable for occurrence surfaceText when verse text is unavailable.
+ */
+function surfaceTextFromLexemes(cluster: ClusterData): string {
+  /* c8 ignore start -- defensive; no cluster with empty lexemes reaches this from conversion */
+  if (cluster.lexemes.length === 0) return '';
+  /* c8 ignore end */
+  return cluster.lexemes
+    .map((l) => {
+      const colon = l.lexemeId.indexOf(':');
+      return colon >= 0 ? l.lexemeId.slice(colon + 1) : l.lexemeId;
+    })
+    .join('');
+}
+
+/**
+ * Groups clusters by text range so that Word and WordParse clusters for the same span are combined
+ * into one occurrence (surface from Word, analyses from WordParse when present). See
+ * data-model.md.
+ */
+function clustersByRange(
+  verseData: VerseData,
+): Map {
+  const byRange = new Map();
+  verseData.clusters.forEach((cluster) => {
+    const key = `${cluster.textRange.index}-${cluster.textRange.length}`;
+    let entry = byRange.get(key);
+    if (!entry) {
+      entry = {};
+      byRange.set(key, entry);
+    }
+    const kind = clusterKind(cluster);
+    if (kind === 'word' && !entry.word) entry.word = cluster;
+    else if (kind === 'wordParse' && !entry.wordParse) entry.wordParse = cluster;
+    // If multiple Word or WordParse for same range, first wins (matches Paratext "best parse" idea).
+  });
+  return byRange;
+}
+
+/**
+ * Converts a Paratext 9 verse to an interlinearizer segment.
+ *
+ * Word and WordParse clusters that share the same text range are merged into a single occurrence:
+ * surface form from the Word cluster when present, analyses from the WordParse cluster when present
+ * (so stem+affix is shown without duplicating the word). See data-model.md.
+ *
+ * @param verseRef - Verse reference (e.g., "MAT 1:1").
+ * @param verseData - Verse data from Paratext 9.
+ * @param glossLanguage - Gloss language code (used for analysis IDs).
+ * @returns A Segment with occurrences in text order (one per word span, one per punctuation).
+ */
+/** Item in the merged list of word spans and punctuations for reading-order sort. */
+type WordSpanOrPunctuation =
+  | {
+      kind: 'word';
+      textRange: StringRange;
+      wordCluster?: ClusterData;
+      wordParseCluster?: ClusterData;
+    }
+  | { kind: 'punctuation'; textRange: StringRange; punctuation: PunctuationData };
+
+/**
+ * Builds a source-language segment: Word:-prefixed lexemes map to occurrence.surfaceText only;
+ * Stem/Suffix/Prefix map to morph analyses linked via assignments.
+ */
+function convertVerseToSourceSegment(
+  verseRef: string,
+  verseData: VerseData,
+  glossLanguage: string,
+): Segment {
+  const segmentId = generateSegmentId(verseRef);
+  const byRange = clustersByRange(verseData);
+
+  const wordSpans: WordSpanOrPunctuation[] = Array.from(byRange.entries())
+    .filter(([, entry]) => entry.word !== undefined || entry.wordParse !== undefined)
+    .map(([rangeKey, entry]) => {
+      const [indexStr, lengthStr] = rangeKey.split('-');
+      const textRange: StringRange = {
+        index: parseInt(indexStr, 10),
+        length: parseInt(lengthStr, 10),
+      };
+      return {
+        kind: 'word' as const,
+        textRange,
+        wordCluster: entry.word,
+        wordParseCluster: entry.wordParse,
+      };
+    });
+
+  const items: WordSpanOrPunctuation[] = [
+    ...wordSpans,
+    ...verseData.punctuations.map(
+      (p): WordSpanOrPunctuation => ({
+        kind: 'punctuation',
+        textRange: p.textRange,
+        punctuation: p,
+      }),
+    ),
+  ].sort((a, b) => {
+    const byIndex = a.textRange.index - b.textRange.index;
+    if (byIndex !== 0) return byIndex;
+    const byLength = a.textRange.length - b.textRange.length;
+    if (byLength !== 0) return byLength;
+    // Word before punctuation when same range (display order: word then following punctuation).
+    if (a.kind !== b.kind) return a.kind === 'word' ? -1 : 1;
+    return 0;
+  });
+
+  const occurrences: Occurrence[] = items.map((item, occurrenceIndex): Occurrence => {
+    if (item.kind === 'word') {
+      const clusterForAssignments = item.wordParseCluster ?? item.wordCluster;
+      const clusterForSurface = item.wordCluster ?? item.wordParseCluster;
+      const singleCluster = clusterForAssignments === clusterForSurface;
+      if (singleCluster) {
+        // Single cluster for this span (Word only or WordParse only). Use it for both assignments and surface.
+        const single = clusterForAssignments ?? clusterForSurface;
+        if (!single) throw new Error('word span with no cluster');
+        const occurrenceId = generateOccurrenceIdFromCluster(segmentId, single.id, occurrenceIndex);
+        const surfaceText = surfaceTextFromLexemes(single);
+        const assignments = single.lexemes
+          .filter((lexeme) => isWordParseLexemeId(lexeme.lexemeId))
+          .map((lexeme): AnalysisAssignment => {
+            const analysisId = generateAnalysisId(lexeme.lexemeId, lexeme.senseId, glossLanguage);
+            return {
+              id: generateAssignmentId(occurrenceId, analysisId),
+              occurrenceId,
+              analysisId,
+              status: verseData.hash ? AssignmentStatus.Approved : AssignmentStatus.Suggested,
+            };
+          });
+        return {
+          id: occurrenceId,
+          segmentId,
+          index: occurrenceIndex,
+          anchor: textRangeToAnchor(single.textRange),
+          surfaceText,
+          writingSystem: '',
+          type: OccurrenceType.Word,
+          assignments,
+        };
+      }
+      if (!clusterForAssignments || !clusterForSurface)
+        throw new Error('word span with no cluster');
+      const occurrenceId = generateOccurrenceIdFromCluster(
+        segmentId,
+        clusterForAssignments.id,
+        occurrenceIndex,
+      );
+      const surfaceText = surfaceTextFromLexemes(clusterForSurface);
+      const assignments = clusterForAssignments.lexemes
+        .filter((lexeme) => isWordParseLexemeId(lexeme.lexemeId))
+        .map((lexeme): AnalysisAssignment => {
+          const analysisId = generateAnalysisId(lexeme.lexemeId, lexeme.senseId, glossLanguage);
+          return {
+            id: generateAssignmentId(occurrenceId, analysisId),
+            occurrenceId,
+            analysisId,
+            status: verseData.hash ? AssignmentStatus.Approved : AssignmentStatus.Suggested,
+          };
+        });
+      return {
+        id: occurrenceId,
+        segmentId,
+        index: occurrenceIndex,
+        anchor: textRangeToAnchor(item.textRange),
+        surfaceText,
+        writingSystem: '',
+        type: OccurrenceType.Word,
+        assignments,
+      };
+    }
+    const { punctuation } = item;
+    return {
+      id: generateOccurrenceIdFromPunctuation(segmentId, punctuation.textRange, occurrenceIndex),
+      segmentId,
+      index: occurrenceIndex,
+      anchor: textRangeToAnchor(punctuation.textRange),
+      surfaceText: punctuation.afterText || punctuation.beforeText || '',
+      writingSystem: '',
+      type: OccurrenceType.Punctuation,
+      assignments: [],
+    };
+  });
+
+  return {
+    id: segmentId,
+    segmentRef: verseRef,
+    baselineText: '', // Paratext 9 doesn't specify baseline text
+    occurrences,
+  };
+}
+
+/** Target segment ID suffix so target and source segments can be matched. */
+const TARGET_SEGMENT_ID_SUFFIX = '-target';
+
+/**
+ * Builds a target-language segment aligned to the source: same order and count of occurrences.
+ * Word-level gloss from Lexicon Word entries → occurrence.surfaceText; Stem/Suffix/Prefix from
+ * Lexicon → morph analyses linked via assignments (target analysis IDs).
+ *
+ * @param verseRef - Verse reference (e.g. "JHN 1:1").
+ * @param verseData - Verse data from Paratext 9.
+ * @param sourceSegmentId - Source segment ID (used to derive target segment and occurrence IDs).
+ * @param glossLanguage - Gloss language code.
+ * @param lexicon - Parsed Lexicon for word-level and sense gloss lookup.
+ * @returns Target Segment with same segmentRef and occurrence count as source.
+ */
+function convertVerseToTargetSegment(
+  verseRef: string,
+  verseData: VerseData,
+  sourceSegmentId: string,
+  glossLanguage: string,
+  lexicon: LexiconData,
+): Segment {
+  const targetSegmentId = sourceSegmentId + TARGET_SEGMENT_ID_SUFFIX;
+  const byRange = clustersByRange(verseData);
+  const glossLookup = buildGlossLookupFromLexicon(lexicon);
+
+  const wordSpans: WordSpanOrPunctuation[] = Array.from(byRange.entries())
+    .filter(([, entry]) => entry.word !== undefined || entry.wordParse !== undefined)
+    .map(([rangeKey, entry]) => {
+      const [indexStr, lengthStr] = rangeKey.split('-');
+      const textRange: StringRange = {
+        index: parseInt(indexStr, 10),
+        length: parseInt(lengthStr, 10),
+      };
+      return {
+        kind: 'word' as const,
+        textRange,
+        wordCluster: entry.word,
+        wordParseCluster: entry.wordParse,
+      };
+    });
+
+  const items: WordSpanOrPunctuation[] = [
+    ...wordSpans,
+    ...verseData.punctuations.map(
+      (p): WordSpanOrPunctuation => ({
+        kind: 'punctuation',
+        textRange: p.textRange,
+        punctuation: p,
+      }),
+    ),
+  ].sort((a, b) => {
+    const byIndex = a.textRange.index - b.textRange.index;
+    if (byIndex !== 0) return byIndex;
+    const byLength = a.textRange.length - b.textRange.length;
+    if (byLength !== 0) return byLength;
+    if (a.kind !== b.kind) return a.kind === 'word' ? -1 : 1;
+    return 0;
+  });
+
+  const occurrences: Occurrence[] = items.map((item, occurrenceIndex): Occurrence => {
+    if (item.kind === 'word') {
+      const clusterForAssignments = item.wordParseCluster ?? item.wordCluster;
+      const clusterForSurface = item.wordCluster ?? item.wordParseCluster;
+      const singleCluster = clusterForAssignments === clusterForSurface;
+      const cluster = singleCluster
+        ? (clusterForAssignments ?? clusterForSurface)
+        : clusterForAssignments;
+      const surfaceCluster = singleCluster ? cluster : clusterForSurface;
+      if (!cluster || !surfaceCluster) throw new Error('word span with no cluster');
+
+      const surfaceForm = surfaceTextFromLexemes(surfaceCluster);
+      // Prefer gloss from Interlinear GlossId (senseId) so the assigned sense is shown; fallback to
+      // Lexicon form-based lookup (e.g. for WordParse-only spans or when sense is missing).
+      const senseGloss = surfaceCluster.lexemes
+        .map((l) => glossLookup(l.senseId, glossLanguage))
+        .find((g) => g !== undefined);
+      const wordLevelGloss = getWordLevelGlossForForm(lexicon, surfaceForm, glossLanguage);
+      const targetSurfaceText = (senseGloss !== undefined ? senseGloss : wordLevelGloss) ?? '';
+
+      const targetOccurrenceId = `${targetSegmentId}-occ-${occurrenceIndex}-${cluster.id}`;
+      const assignments = cluster.lexemes
+        .filter((lexeme) => isWordParseLexemeId(lexeme.lexemeId))
+        .map((lexeme): AnalysisAssignment => {
+          const analysisId =
+            TARGET_ANALYSIS_ID_PREFIX +
+            generateAnalysisId(lexeme.lexemeId, lexeme.senseId, glossLanguage);
+          return {
+            id: generateAssignmentId(targetOccurrenceId, analysisId),
+            occurrenceId: targetOccurrenceId,
+            analysisId,
+            status: verseData.hash ? AssignmentStatus.Approved : AssignmentStatus.Suggested,
+          };
+        });
+
+      return {
+        id: targetOccurrenceId,
+        segmentId: targetSegmentId,
+        index: occurrenceIndex,
+        anchor: textRangeToAnchor(cluster.textRange),
+        surfaceText: targetSurfaceText,
+        writingSystem: '',
+        type: OccurrenceType.Word,
+        assignments,
+      };
+    }
+    const { punctuation } = item;
+    const targetPuncId = `${targetSegmentId}-punc-${occurrenceIndex}-${punctuation.textRange.index}-${punctuation.textRange.length}`;
+    return {
+      id: targetPuncId,
+      segmentId: targetSegmentId,
+      index: occurrenceIndex,
+      anchor: textRangeToAnchor(punctuation.textRange),
+      surfaceText: punctuation.afterText || punctuation.beforeText || '',
+      writingSystem: '',
+      type: OccurrenceType.Punctuation,
+      assignments: [],
+    };
+  });
+
+  return {
+    id: targetSegmentId,
+    segmentRef: verseRef,
+    baselineText: '',
+    occurrences,
+  };
+}
+
+/** Options for {@link createSourceAnalyses} (no options) and {@link createTargetAnalyses}. */
+export type CreateTargetAnalysesOptions = {
+  /** Lookup (senseId, language) → gloss text from Lexicon. */
+  glossLookup?: LexiconGlossLookup;
+};
+
+/**
+ * Options for legacy {@link createAnalyses}. Prefer {@link createSourceAnalyses} and
+ * {@link createTargetAnalyses} with separate source/target interlinearizations.
+ */
+export type CreateAnalysesOptions = {
+  /** Lookup (senseId, language) → gloss text. Used for combined single interlinearization. */
+  glossLookup?: LexiconGlossLookup;
+};
+
+/**
+ * Creates source-side Analysis objects: only WordParse lexemes (Stem/Suffix/Prefix) as Morph
+ * analyses with form in morphemeBundles. Word:-prefixed lexemes are not analyses; they map to
+ * occurrence.surfaceText only.
+ *
+ * @param interlinearData - Paratext 9 interlinear data.
+ * @returns Map of source analysis ID to Analysis (Morph type).
+ */
+export function createSourceAnalyses(interlinearData: InterlinearData): Map {
+  const analyses = new Map();
+  const { glossLanguage } = interlinearData;
+
+  Object.values(interlinearData.verses).forEach((verseData) => {
+    verseData.clusters.forEach((cluster) => {
+      cluster.lexemes
+        .filter((lexeme) => isWordParseLexemeId(lexeme.lexemeId))
+        .forEach((lexeme) => {
+          const analysisId = generateAnalysisId(lexeme.lexemeId, lexeme.senseId, glossLanguage);
+          if (analyses.has(analysisId)) return;
+
+          const displayForm = sourceMorphDisplayForm(lexeme.lexemeId);
+          analyses.set(analysisId, {
+            id: analysisId,
+            analysisLanguage: glossLanguage,
+            analysisType: AnalysisType.Morph,
+            confidence: Confidence.Medium,
+            sourceSystem: 'paratext-9',
+            sourceUser: 'paratext-9-parser',
+            morphemeBundles: [
+              {
+                id: `${analysisId}-bundle-0`,
+                index: 0,
+                form: displayForm,
+                writingSystem: '',
+              },
+            ],
+          });
+        });
+    });
+  });
+
+  return analyses;
+}
+
+/**
+ * Creates target-side Analysis objects: only WordParse lexemes as Morph analyses with glossText
+ * from the Lexicon. IDs are prefixed so they do not collide with source analyses.
+ *
+ * @param interlinearData - Paratext 9 interlinear data.
+ * @param options - GlossLookup for senseId → gloss text.
+ * @returns Map of target analysis ID to Analysis (Morph type with glossText).
+ */
+export function createTargetAnalyses(
+  interlinearData: InterlinearData,
+  options?: CreateTargetAnalysesOptions,
+): Map {
+  const analyses = new Map();
+  const { glossLanguage } = interlinearData;
+  const glossLookup = options?.glossLookup;
+
+  Object.values(interlinearData.verses).forEach((verseData) => {
+    verseData.clusters.forEach((cluster) => {
+      cluster.lexemes
+        .filter((lexeme) => isWordParseLexemeId(lexeme.lexemeId))
+        .forEach((lexeme) => {
+          const analysisId =
+            TARGET_ANALYSIS_ID_PREFIX +
+            generateAnalysisId(lexeme.lexemeId, lexeme.senseId, glossLanguage);
+          if (analyses.has(analysisId)) return;
+
+          let glossText: string | undefined;
+          if (glossLookup && lexeme.senseId) {
+            const fromLexicon = glossLookup(lexeme.senseId, glossLanguage);
+            glossText = fromLexicon !== undefined ? fromLexicon : lexeme.senseId;
+          } else {
+            glossText = lexeme.senseId || undefined;
+          }
+
+          const displayForm = sourceMorphDisplayForm(lexeme.lexemeId);
+          analyses.set(analysisId, {
+            id: analysisId,
+            analysisLanguage: glossLanguage,
+            analysisType: AnalysisType.Morph,
+            confidence: Confidence.Medium,
+            sourceSystem: 'paratext-9',
+            sourceUser: 'paratext-9-parser',
+            glossText,
+            morphemeBundles: [
+              {
+                id: `${analysisId}-bundle-0`,
+                index: 0,
+                form: displayForm,
+                writingSystem: '',
+              },
+            ],
+          });
+        });
+    });
+  });
+
+  return analyses;
+}
+
+/**
+ * Legacy: creates a single combined analysis map (source morph analyses only). Prefer
+ * createSourceAnalyses + createTargetAnalyses with InterlinearAlignment.
+ *
+ * @param interlinearData - Paratext 9 interlinear data.
+ * @param options - Optional. glossLookup (ignored for source-only; kept for API compat).
+ * @returns Map of analysis ID to Analysis (source morph only).
+ */
+export function createAnalyses(interlinearData: InterlinearData): Map {
+  return createSourceAnalyses(interlinearData);
+}
+
+/** Options for Paratext 9 conversion (source-only or full alignment). */
+export type ConvertParatext9Options = {
+  /** SHA-256 hex hasher for composite book text version. Default: Web Crypto (WebView-safe). */
+  hashSha256Hex?: (input: string) => Promise;
+};
+
+/**
+ * Converts Paratext 9 InterlinearData to the **source** interlinearization only. Word:-prefixed
+ * lexemes map to occurrence.surfaceText; Stem/Suffix/Prefix map to morph analyses via assignments.
+ * Use {@link convertParatext9ToInterlinearAlignment} when you have a Lexicon and want source +
+ * target in one structure.
+ *
+ * @param interlinearData - Paratext 9 interlinear data to convert.
+ * @param options - Optional. hashSha256Hex for book-level text version.
+ * @returns Promise that resolves to the source Interlinearization.
+ */
+export async function convertParatext9ToInterlinearization(
+  interlinearData: InterlinearData,
+  options?: ConvertParatext9Options,
+): Promise {
+  const { glossLanguage, bookId, verses } = interlinearData;
+  const hashSha256Hex = options?.hashSha256Hex ?? sha256HexWebCrypto;
+
+  const interlinearizationId = generateInterlinearizationId(bookId);
+  const analyzedBookId = generateBookId(bookId);
+  const sortedVerseRefs = Object.keys(verses).sort();
+  const verseDataArray = sortedVerseRefs.map((ref) => verses[ref]);
+  const segments = sortedVerseRefs.map((ref) =>
+    convertVerseToSourceSegment(ref, verses[ref], glossLanguage),
+  );
+
+  const textVersion = await computeBookTextVersion(verseDataArray, hashSha256Hex);
+
+  const analyzedBook: AnalyzedBook = {
+    id: analyzedBookId,
+    bookRef: bookId,
+    textVersion,
+    segments,
+  };
+
+  return {
+    id: interlinearizationId,
+    sourceWritingSystem: '',
+    analysisLanguages: [glossLanguage],
+    books: [analyzedBook],
+  };
+}
+
+/**
+ * Builds the target interlinearization from the source and Lexicon: same segment/occurrence layout;
+ * occurrence.surfaceText = word-level gloss from Lexicon Word entries; assignments = morph analyses
+ * (Stem/Suffix/Prefix) with glossText from Lexicon.
+ */
+function buildTargetInterlinearization(
+  source: Interlinearization,
+  interlinearData: InterlinearData,
+  lexicon: LexiconData,
+): Interlinearization {
+  const { glossLanguage, verses } = interlinearData;
+  const targetId = source.id + TARGET_SEGMENT_ID_SUFFIX;
+
+  const books: AnalyzedBook[] = source.books.map((book) => {
+    const targetBookId = book.id + TARGET_SEGMENT_ID_SUFFIX;
+    const segments: Segment[] = book.segments.map((segment) => {
+      const verseRef = segment.segmentRef;
+      const verseData = verses[verseRef];
+      if (!verseData) {
+        return {
+          id: segment.id + TARGET_SEGMENT_ID_SUFFIX,
+          segmentRef: verseRef,
+          baselineText: '',
+          occurrences: segment.occurrences.map((occ) => ({
+            ...occ,
+            id: occ.id + TARGET_SEGMENT_ID_SUFFIX,
+            segmentId: segment.id + TARGET_SEGMENT_ID_SUFFIX,
+            surfaceText: '',
+            assignments: [],
+          })),
+        };
+      }
+      return convertVerseToTargetSegment(verseRef, verseData, segment.id, glossLanguage, lexicon);
+    });
+    return {
+      id: targetBookId,
+      bookRef: book.bookRef,
+      textVersion: book.textVersion,
+      segments,
+    };
+  });
+
+  return {
+    id: targetId,
+    sourceWritingSystem: '',
+    analysisLanguages: [glossLanguage],
+    books,
+  };
+}
+
+/**
+ * Converts Paratext 9 InterlinearData + optional Lexicon to an InterlinearAlignment with separate
+ * source and target interlinearizations. Source: from Interlinear XML (Word → surfaceText,
+ * Stem/Suffix/Prefix → analyses). Target: from Lexicon (Word → surfaceText, other types →
+ * analyses), aligned by segment and occurrence index.
+ *
+ * @param interlinearData - Paratext 9 interlinear data.
+ * @param lexicon - Optional. When provided, target occurrence.surfaceText and target analyses get
+ *   gloss text from the Lexicon.
+ * @param options - Optional. hashSha256Hex for book-level text version.
+ * @returns Promise that resolves to InterlinearAlignment (source, target, links).
+ */
+export async function convertParatext9ToInterlinearAlignment(
+  interlinearData: InterlinearData,
+  lexicon: LexiconData | undefined,
+  options?: ConvertParatext9Options,
+): Promise {
+  const source = await convertParatext9ToInterlinearization(interlinearData, options);
+  const target = lexicon
+    ? buildTargetInterlinearization(source, interlinearData, lexicon)
+    : buildTargetInterlinearizationEmpty(source);
+
+  return {
+    id: `${source.id}-alignment`,
+    source,
+    target,
+    links: [],
+  };
+}
+
+/**
+ * Builds a target interlinearization with same structure as source but empty surfaceText and no
+ * assignments (when no Lexicon is provided).
+ */
+function buildTargetInterlinearizationEmpty(source: Interlinearization): Interlinearization {
+  const targetId = source.id + TARGET_SEGMENT_ID_SUFFIX;
+
+  const books: AnalyzedBook[] = source.books.map((book) => ({
+    id: book.id + TARGET_SEGMENT_ID_SUFFIX,
+    bookRef: book.bookRef,
+    textVersion: book.textVersion,
+    segments: book.segments.map((segment) => ({
+      id: segment.id + TARGET_SEGMENT_ID_SUFFIX,
+      segmentRef: segment.segmentRef,
+      baselineText: '',
+      occurrences: segment.occurrences.map((occ) => ({
+        ...occ,
+        id: occ.id + TARGET_SEGMENT_ID_SUFFIX,
+        segmentId: segment.id + TARGET_SEGMENT_ID_SUFFIX,
+        surfaceText: '',
+        assignments: [],
+      })),
+    })),
+  }));
+
+  return {
+    id: targetId,
+    sourceWritingSystem: '',
+    analysisLanguages: source.analysisLanguages,
+    books,
+  };
+}
diff --git a/src/parsers/interlinearXmlParser.ts b/src/parsers/paratext-9/interlinearParser.ts
similarity index 84%
rename from src/parsers/interlinearXmlParser.ts
rename to src/parsers/paratext-9/interlinearParser.ts
index 7102b929..eb1ac02e 100644
--- a/src/parsers/interlinearXmlParser.ts
+++ b/src/parsers/paratext-9/interlinearParser.ts
@@ -6,7 +6,7 @@ import type {
   StringRange,
   InterlinearData,
   VerseData,
-} from 'interlinearizer';
+} from './types';
 
 /** Range: Index and Length attributes. */
 interface ParsedRange {
@@ -30,7 +30,7 @@ interface ParsedCluster {
   Range?: ParsedRange;
   /** Lexeme elements in this cluster. */
   Lexeme?: ParsedLexeme[];
-  /** Excluded flag (optional). See [pt9-xml.md](pt9-xml.md) for details. */
+  /** Excluded flag (optional). See [xml-schema.md](xml-schema.md) for details. */
   Excluded?: string;
 }
 
@@ -62,10 +62,8 @@ interface ParsedVersesItem {
   VerseData?: ParsedVerseData;
 }
 
-/** Root element: ScrTextName, GlossLanguage, BookId, Verses (with item[]). */
+/** Root element: GlossLanguage, BookId, Verses (with item[]). */
 interface ParsedInterlinearDataRoot {
-  /** Source text name (FXP attribute ScrTextName). */
-  ['@_ScrTextName']?: string;
   /** Gloss language (FXP attribute GlossLanguage). */
   ['@_GlossLanguage']?: string;
   /** Book id (FXP attribute BookId). */
@@ -95,7 +93,7 @@ function extractLexemesFromCluster(clusterElement: ParsedCluster): LexemeData[]
     if (!lexemeId) {
       throw new Error('Invalid XML: Lexeme missing required Id attribute');
     }
-    return { LexemeId: lexemeId, SenseId: el['@_GlossId'] ?? '' };
+    return { lexemeId, senseId: el['@_GlossId'] ?? '' };
   });
 }
 
@@ -122,9 +120,9 @@ function extractPunctuationsFromVerse(verseDataElement: ParsedVerseData): Punctu
     if (!Number.isFinite(index) || !Number.isFinite(length)) return [];
     return [
       {
-        TextRange: { Index: index, Length: length },
-        BeforeText: el.BeforeText ?? '',
-        AfterText: el.AfterText ?? '',
+        textRange: { index, length },
+        beforeText: el.BeforeText ?? '',
+        afterText: el.AfterText ?? '',
       },
     ];
   });
@@ -153,21 +151,21 @@ function extractClustersFromVerse(verseDataElement: ParsedVerseData): ClusterDat
       throw new Error('Invalid XML: Range missing required Index or Length attributes');
     }
 
-    const textRange: StringRange = { Index: index, Length: length };
+    const textRange: StringRange = { index, length };
     const lexemes = extractLexemesFromCluster(el);
 
     // Join with "/"; lexeme IDs may contain "/", so do not split LexemesId elsewhere.
-    const lexemesId = lexemes.map((l) => l.LexemeId).join('/');
+    const lexemesId = lexemes.map((l) => l.lexemeId).join('/');
     /** Cluster Id: LexemesId/Index-Length when lexemes present; Index-Length when none. */
     const id = lexemesId ? `${lexemesId}/${index}-${length}` : `${index}-${length}`;
     const excluded = el.Excluded === 'true';
 
     return {
-      TextRange: textRange,
-      Lexemes: lexemes,
-      LexemesId: lexemesId,
-      Id: id,
-      Excluded: excluded,
+      textRange,
+      lexemes,
+      lexemesId,
+      id,
+      excluded,
     };
   });
 }
@@ -176,10 +174,10 @@ function extractClustersFromVerse(verseDataElement: ParsedVerseData): ClusterDat
  * Parses interlinear XML strings into {@link InterlinearData} using fast-xml-parser.
  *
  * Input is a raw XML string (caller is responsible for obtaining it, e.g. from file or network).
- * Output matches the types in `interlinearizer`; no extra conversion is done. Expects the
- * interlinear XML schema described in [pt9-xml.md](pt9-xml.md).
+ * Output matches the types in `types`; no extra conversion is done. Expects the Paratext 9
+ * Interlinear XML schema described in [xml-schema.md](xml-schema.md).
  */
-export class InterlinearXmlParser {
+export class Paratext9Parser {
   private readonly parser: XMLParser;
 
   /**
@@ -213,8 +211,8 @@ export class InterlinearXmlParser {
    * @param xml - Raw XML string (e.g. file contents). Must be valid interlinear XML with
    *   InterlinearData root, GlossLanguage and BookId attributes, and Verses containing item
    *   entries.
-   * @returns Parsed interlinear data: ScrTextName, GlossLanguage, BookId, and Verses (record of
-   *   verse key to {@link VerseData} with Hash, Clusters, Punctuations).
+   * @returns Parsed interlinear data: GlossLanguage, BookId, and Verses (record of verse key to
+   *   {@link VerseData} with Hash, Clusters, Punctuations).
    * @throws {Error} If the root element, required attributes (GlossLanguage, BookId), required
    *   structure (Verses, Cluster Range, Lexeme Id), or duplicate verse reference is present.
    */
@@ -225,7 +223,6 @@ export class InterlinearXmlParser {
       throw new Error('Invalid XML: Missing InterlinearData root element');
     }
 
-    const scrTextName = root['@_ScrTextName'] ?? '';
     const glossLanguage = root['@_GlossLanguage'] ?? '';
     const bookId = root['@_BookId'] ?? '';
     if (!glossLanguage || !bookId) {
@@ -251,23 +248,22 @@ export class InterlinearXmlParser {
 
       const verseDataElement = item.VerseData;
       if (!verseDataElement) {
-        acc[verseKey] = { Hash: '', Clusters: [], Punctuations: [] };
+        acc[verseKey] = { hash: '', clusters: [], punctuations: [] };
         return acc;
       }
 
       acc[verseKey] = {
-        Hash: verseDataElement['@_Hash'] ?? '',
-        Clusters: extractClustersFromVerse(verseDataElement),
-        Punctuations: extractPunctuationsFromVerse(verseDataElement),
+        hash: verseDataElement['@_Hash'] ?? '',
+        clusters: extractClustersFromVerse(verseDataElement),
+        punctuations: extractPunctuationsFromVerse(verseDataElement),
       };
       return acc;
     }, {});
 
     return {
-      ScrTextName: scrTextName,
-      GlossLanguage: glossLanguage,
-      BookId: bookId,
-      Verses: verses,
+      glossLanguage,
+      bookId,
+      verses,
     };
   }
 }
diff --git a/src/parsers/paratext-9/lexiconParser.ts b/src/parsers/paratext-9/lexiconParser.ts
new file mode 100644
index 00000000..a84fbcc1
--- /dev/null
+++ b/src/parsers/paratext-9/lexiconParser.ts
@@ -0,0 +1,253 @@
+/**
+ * @file Parses Paratext 9 Lexicon XML into a structure aligned with Paratext LexiconData
+ *   (LexiconData.cs). Provides (senseId, language) → gloss lookup and word-level gloss lookup for a
+ *   word form (from Word entries) for use in PT9 → interlinearizer conversion.
+ */
+
+import { X2jOptions, XMLParser } from 'fast-xml-parser';
+import type {
+  LexiconData,
+  LexiconEntry,
+  LexiconGloss,
+  LexiconSense,
+  LexemeKey,
+  LexemeType,
+} from './types';
+
+/** Separator used in the internal gloss map key (senseId + language). */
+const GLOSS_KEY_SEP = '\t';
+
+/** Language used when no Language attribute is set; fallback in lookup. */
+const DEFAULT_LANGUAGE = '*';
+
+/** Lexeme Id from key. Matches Paratext LexemeKey.Id: "Type:LexicalForm" or "Type:LexicalForm:H". */
+export function lexemeKeyId(key: LexemeKey): string {
+  if (key.homograph <= 1) return `${key.type}:${key.lexicalForm}`;
+  return `${key.type}:${key.lexicalForm}:${key.homograph}`;
+}
+
+/** Lookup: (senseId, language) → gloss text. Empty string when sense exists but gloss blank. */
+export type LexiconGlossLookup = (senseId: string, language: string) => string | undefined;
+
+/**
+ * Lookup: (wordForm, language) → word-level gloss when Lexicon has a Word entry for that form. Used
+ * for WordParse-only occurrences so the Gloss row shows the surface word gloss from the Lexicon
+ * instead of leaving it blank.
+ */
+export type WordLevelGlossLookup = (wordForm: string, language: string) => string | undefined;
+
+/** Parsed Gloss element. */
+interface ParsedGloss {
+  ['@_Language']?: string;
+  ['#text']?: string;
+}
+
+/** Parsed Sense element. */
+interface ParsedSense {
+  ['@_Id']?: string;
+  Gloss?: ParsedGloss | string | (ParsedGloss | string)[];
+}
+
+/** Parsed Entry. */
+interface ParsedEntry {
+  Sense?: ParsedSense | ParsedSense[];
+}
+
+/** Parsed Entries.item (Lexeme + Entry). */
+interface ParsedLexiconItem {
+  Lexeme?: {
+    ['@_Type']?: string;
+    ['@_Form']?: string;
+    ['@_Homograph']?: string;
+  };
+  Entry?: ParsedEntry;
+}
+
+/** Parsed root. */
+interface ParsedLexiconRoot {
+  Language?: string;
+  FontName?: string;
+  FontSize?: number;
+  Entries?: { item?: ParsedLexiconItem | ParsedLexiconItem[] };
+  Analyses?: unknown;
+}
+
+interface ParsedLexiconXml {
+  Lexicon?: ParsedLexiconRoot;
+}
+
+export function toArray(value: T | T[] | undefined): T[] {
+  if (value === undefined) return [];
+  return Array.isArray(value) ? value : [value];
+}
+
+function glossKey(senseId: string, language: string): string {
+  return `${senseId}${GLOSS_KEY_SEP}${language}`;
+}
+
+export function normalizeGloss(gloss: ParsedGloss | string): { lang: string; text: string } {
+  if (typeof gloss === 'string') {
+    return { lang: DEFAULT_LANGUAGE, text: gloss };
+  }
+  const lang = String(gloss['@_Language'] ?? '') || DEFAULT_LANGUAGE;
+  const text = String(gloss['#text'] ?? '');
+  return { lang, text };
+}
+
+function parseSense(sense: ParsedSense): LexiconSense | undefined {
+  const id = sense['@_Id'] ?? '';
+  if (!id) return undefined;
+  const glosses: LexiconGloss[] = toArray(sense.Gloss).map((g) => {
+    const { lang, text } = normalizeGloss(g);
+    return { language: lang, text };
+  });
+  return { id, glosses };
+}
+
+const VALID_LEXEME_TYPES: readonly LexemeType[] = [
+  'Word',
+  'Stem',
+  'Suffix',
+  'Prefix',
+  'Infix',
+  'Lemma',
+  'Phrase',
+];
+
+function parseLexemeType(s: string): LexemeType {
+  const i = VALID_LEXEME_TYPES.findIndex((t) => t === s);
+  return i >= 0 ? VALID_LEXEME_TYPES[i] : 'Word';
+}
+
+function parseItem(item: ParsedLexiconItem): LexiconEntry | undefined {
+  const lex = item.Lexeme;
+  const entry = item.Entry;
+  if (!lex || !entry) return undefined;
+  const typeStr = String(lex['@_Type'] ?? 'Word').trim() || 'Word';
+  const type = parseLexemeType(typeStr);
+  const lexicalForm = String(lex['@_Form'] ?? '').trim();
+  const homograph = Math.max(1, parseInt(String(lex['@_Homograph'] ?? '1'), 10) || 1);
+  const key: LexemeKey = { type, lexicalForm, homograph };
+  const senses: LexiconSense[] = toArray(entry.Sense)
+    .map(parseSense)
+    .filter((s): s is LexiconSense => s !== undefined);
+  return { key, senses };
+}
+
+/**
+ * Parses Lexicon XML into LexiconData (PT9-aligned). Entries are keyed by lexeme Id (e.g.
+ * "Word:beginning", "Stem:begin").
+ *
+ * @param xml - Raw Lexicon XML string.
+ * @returns LexiconData with entries keyed by lexeme Id.
+ * @throws {Error} If root element is not Lexicon.
+ */
+export function parseLexicon(xml: string): LexiconData {
+  const arrayPaths = new Set([
+    'Lexicon.Entries.item',
+    'Lexicon.Entries.item.Entry.Sense',
+    'Lexicon.Entries.item.Entry.Sense.Gloss',
+  ]);
+  const options: Partial = {
+    ignoreAttributes: false,
+    attributeNamePrefix: '@_',
+    ignoreDeclaration: true,
+    ignorePiTags: true,
+    trimValues: false,
+    parseTagValue: false,
+    isArray: (_tagName, jPath) => arrayPaths.has(jPath),
+  };
+  const parser = new XMLParser(options);
+  const parsed: ParsedLexiconXml = parser.parse(xml);
+  const root = parsed.Lexicon;
+  if (!root) {
+    throw new Error('Invalid XML: Missing Lexicon root element');
+  }
+
+  const entries: Record = {};
+  toArray(root.Entries?.item).forEach((item) => {
+    const lexiconEntry = parseItem(item);
+    if (!lexiconEntry) return;
+    const id = lexemeKeyId(lexiconEntry.key);
+    entries[id] = lexiconEntry;
+  });
+
+  return {
+    language: String(root.Language ?? '').trim(),
+    fontName: String(root.FontName ?? 'Arial').trim() || 'Arial',
+    fontSize: Math.max(0, parseInt(String(root.FontSize ?? '10'), 10) || 10),
+    entries,
+  };
+}
+
+/**
+ * Returns word-level gloss for a word form when the Lexicon has a Word entry for that form. Uses
+ * first sense and requested language (or default). Used for WordParse-only occurrences.
+ *
+ * @param lexicon - Parsed LexiconData.
+ * @param wordForm - Surface word form (e.g. "beginning").
+ * @param language - Gloss language code.
+ * @returns Gloss text or undefined if no Word entry or no gloss for that language.
+ */
+export function getWordLevelGlossForForm(
+  lexicon: LexiconData,
+  wordForm: string,
+  language: string,
+): string | undefined {
+  const id = `Word:${wordForm}`;
+  const entry = lexicon.entries[id];
+  if (!entry?.senses?.length) return undefined;
+  const sense = entry.senses[0];
+  const exact = sense.glosses.find((g) => g.language === language);
+  if (exact !== undefined) return exact.text;
+  const fallback = sense.glosses.find((g) => !g.language || g.language === DEFAULT_LANGUAGE);
+  return fallback?.text;
+}
+
+/**
+ * Builds a (senseId, language) → gloss lookup from parsed LexiconData. Preserves existing behaviour
+ * for createAnalyses.
+ *
+ * @param lexicon - Parsed LexiconData (from parseLexicon).
+ * @returns LexiconGlossLookup.
+ */
+export function buildGlossLookupFromLexicon(lexicon: LexiconData): LexiconGlossLookup {
+  const glossMap = new Map();
+  Object.values(lexicon.entries).forEach((entry) => {
+    entry.senses.forEach((sense) => {
+      sense.glosses.forEach((g) => {
+        glossMap.set(glossKey(sense.id, g.language), g.text);
+      });
+    });
+  });
+  return (senseId: string, language: string): string | undefined => {
+    if (!senseId) return undefined;
+    const exact = glossMap.get(glossKey(senseId, language));
+    if (exact !== undefined) return exact;
+    return glossMap.get(glossKey(senseId, DEFAULT_LANGUAGE));
+  };
+}
+
+/**
+ * Builds (wordForm, language) → word-level gloss lookup from parsed LexiconData.
+ *
+ * @param lexicon - Parsed LexiconData.
+ * @returns WordLevelGlossLookup.
+ */
+export function buildWordLevelGlossLookup(lexicon: LexiconData): WordLevelGlossLookup {
+  return (wordForm: string, language: string) =>
+    getWordLevelGlossForForm(lexicon, wordForm, language);
+}
+
+/**
+ * Parses Lexicon XML and returns a (senseId, language) → gloss lookup. Kept for backward
+ * compatibility; prefer parseLexicon + buildGlossLookupFromLexicon when you also need word-level
+ * gloss or LexiconData.
+ *
+ * @param xml - Raw Lexicon XML string.
+ * @returns LexiconGlossLookup.
+ */
+export function parseLexiconAndBuildGlossLookup(xml: string): LexiconGlossLookup {
+  const lexicon = parseLexicon(xml);
+  return buildGlossLookupFromLexicon(lexicon);
+}
diff --git a/src/parsers/paratext-9/types.ts b/src/parsers/paratext-9/types.ts
new file mode 100644
index 00000000..e61ce75a
--- /dev/null
+++ b/src/parsers/paratext-9/types.ts
@@ -0,0 +1,128 @@
+/** Character range in source text (Index, Length). */
+export interface StringRange {
+  /** Start index of the range in the source text (0-based). */
+  index: number;
+  /** Number of characters in the range. */
+  length: number;
+}
+
+/** Data on the interlinearization of a single lexeme. */
+export interface LexemeData {
+  /**
+   * Source-word/wordParse ID from Interlinear XML (attribute Id). Identifies the word or morpheme
+   * in the source text at this position (e.g. "Word:In", "Stem:begin"); not from the Lexicon.
+   */
+  lexemeId: string;
+  /**
+   * ID of the target-language sense used for the gloss (Interlinear XML attribute GlossId).
+   * References Lexicon Sense Id; the Lexicon holds the gloss-language words/senses/glosses.
+   */
+  senseId: string;
+}
+
+/** Data on the interlinearization of a cluster. */
+export interface ClusterData {
+  /** Character range this cluster occupies in the verse text. */
+  textRange: StringRange;
+  /** Lexemes in this cluster, in order. */
+  lexemes: LexemeData[];
+  /** Slash-joined LexemeIds for this cluster (e.g. "Word:a/Word:b"). */
+  lexemesId: string;
+  /** Unique cluster id: LexemesId plus TextRange (e.g. "Word:a/Word:b/21-3"). */
+  id: string;
+  /** Excluded flag. See [xml-schema.md](xml-schema.md) for details. */
+  excluded: boolean;
+}
+
+/** Data on punctuation change. */
+export interface PunctuationData {
+  /** Character range this punctuation occupies in the verse text. */
+  textRange: StringRange;
+  /** Punctuation text before the change (or empty). */
+  beforeText: string;
+  /** Punctuation text after the change (or empty). */
+  afterText: string;
+}
+
+/** Interlinear data for a single verse. */
+export interface VerseData {
+  /** Hash of verse text when approved; empty string if not approved. */
+  hash: string;
+  /** Lexeme clusters in this verse. */
+  clusters: ClusterData[];
+  /** Punctuation changes in this verse. */
+  punctuations: PunctuationData[];
+}
+
+/** Root interlinear data: book + verses. */
+export interface InterlinearData {
+  /** Language code or name for the glosses. */
+  glossLanguage: string;
+  /** Book id (e.g. "RUT", "MAT"). */
+  bookId: string;
+  /**
+   * Verse data keyed by verse reference (e.g. "RUT 3:1"). Exactly one entry per reference; the
+   * parser rejects XML that contains duplicate verse references.
+   */
+  verses: Record;
+}
+
+/**
+ * Morphological type of a lexeme. Matches Paratext LexemeType (LexemeKey.Type). Id format:
+ * "Type:LexicalForm" or "Type:LexicalForm:Homograph".
+ */
+export type LexemeType = 'Word' | 'Stem' | 'Suffix' | 'Prefix' | 'Infix' | 'Lemma' | 'Phrase';
+
+/**
+ * Key for a single lexeme in the lexicon. Mirrors Paratext LexemeKey (Type, Form, Homograph). Id =
+ * "Type:LexicalForm" (or "Type:LexicalForm:Homograph" when homograph > 1).
+ */
+export interface LexemeKey {
+  /** Morphological type. */
+  type: LexemeType;
+  /** Lexical form (e.g. stem without affixes, or word). */
+  lexicalForm: string;
+  /** Homograph number; typically 1. */
+  homograph: number;
+}
+
+/** Gloss for a sense in a target language. Mirrors XmlLexiconGloss (Language, Text). */
+export interface LexiconGloss {
+  /** Language code (e.g. "en"). */
+  language: string;
+  /** Gloss text. */
+  text: string;
+}
+
+/** Sense of a lexeme. Mirrors XmlLexiconSense (Id, Glosses). */
+export interface LexiconSense {
+  /** Sense id; matches GlossId in Interlinear XML. */
+  id: string;
+  /** Glosses by language. */
+  glosses: LexiconGloss[];
+}
+
+/** Lexicon entry for one lexeme. Mirrors XmlLexiconEntry (Key → Senses). */
+export interface LexiconEntry {
+  /** Lexeme key (Type, Form, Homograph). */
+  key: LexemeKey;
+  /** Senses for this lexeme. */
+  senses: LexiconSense[];
+}
+
+/** Word form → list of LexemeKeys (morphological analysis). Mirrors LexiconData.Analyses. */
+export type WordAnalyses = Record;
+
+/** Parsed Lexicon. Mirrors Paratext LexiconData: Language, FontName, FontSize, Entries, Analyses. */
+export interface LexiconData {
+  /** Language of the lexicon. */
+  language: string;
+  /** Default font name. */
+  fontName: string;
+  /** Default font size. */
+  fontSize: number;
+  /** Entries keyed by lexeme Id (e.g. "Word:beginning", "Stem:begin"). */
+  entries: Record;
+  /** Optional: word form → morphological analysis (list of lexeme keys). */
+  analyses?: WordAnalyses;
+}
diff --git a/src/parsers/pt9-xml.md b/src/parsers/paratext-9/xml-schema.md
similarity index 64%
rename from src/parsers/pt9-xml.md
rename to src/parsers/paratext-9/xml-schema.md
index 56d0baac..31bdeb69 100644
--- a/src/parsers/pt9-xml.md
+++ b/src/parsers/paratext-9/xml-schema.md
@@ -1,6 +1,8 @@
 # Paratext 9 XML schema
 
-The extension reads PT9 interlinear data from XML files (e.g. `Interlinear__.xml` in project data). The parser in `src/parsers/interlinearXmlParser.ts` expects the following structure. Sample files live in `test-data/` (e.g. `Interlinear_en_MAT.xml`).
+The extension reads PT9 interlinear data from XML files (e.g. `Interlinear__.xml` in project data). The parser in `src/parsers/paratext-9/interlinearParser.ts` expects the following structure. Sample files live in `test-data/` (e.g. `Interlinear_en_JHN.xml`).
+
+Interlinear XML does **not** store gloss strings; it only references senses by `GlossId` (which corresponds to Sense Id in the Lexicon). To populate `glossText` on the interlinearizer model, the extension can load a Paratext 9 **Lexicon** XML file (`Lexicon.xml`). The Lexicon parser (`src/parsers/paratext-9/lexiconParser.ts`) builds a (senseId, language) → gloss text lookup; `createAnalyses` in `converter` accepts an optional `glossLookup` so analyses get real gloss text when the Lexicon is available.
 
 ## Document structure
 
@@ -8,7 +10,6 @@ The extension reads PT9 interlinear data from XML files (e.g. `Interlinear_
-
+
   
     
       RUT 1:1
@@ -116,3 +117,21 @@ This example shows optional root attributes, verse `Hash`, multiple verses and c
   
 
 ```
+
+## Lexicon XML (gloss text lookup)
+
+The Lexicon file (`Lexicon.xml`) provides gloss text per Sense and language. Structure:
+
+- **Root element:** `Lexicon`
+  - **Child:** `Entries` containing `item` elements (one per lexicon entry).
+- **`item`**
+  - **`Lexeme`** (optional): Attributes `Type`, `Form`, `Homograph` (for reference; not used for lookup).
+  - **`Entry`**: Contains **`Sense`** elements.
+- **`Sense`**
+  - **Attribute:** `Id` (required). This Id is the same as `GlossId` in Interlinear XML.
+  - **Children:** One or more **`Gloss`** elements.
+- **`Gloss`**
+  - **Attribute:** `Language` (e.g. `"en"`, `"hbo"`, `"grc"`). The gloss language in InterlinearData should match when resolving word-level glosses.
+  - **Text content:** The gloss string (may be empty).
+
+The converter uses `parseLexiconAndBuildGlossLookup(xml)` to build a function `(senseId, language) => glossText`. Pass that function as `createAnalyses(interlinearData, { glossLookup })` so `Analysis.glossText` is set from the Lexicon instead of the senseId placeholder.
diff --git a/src/types/interlinearizer-enums.ts b/src/types/interlinearizer-enums.ts
new file mode 100644
index 00000000..238baa42
--- /dev/null
+++ b/src/types/interlinearizer-enums.ts
@@ -0,0 +1,58 @@
+/**
+ * @file Runtime enum values for the interlinearizer model.
+ *
+ *   Type declarations (and these enums as types) live in interlinearizer.d.ts for the declared module
+ *   'interlinearizer'. This file provides the actual enum values so code that imports from this
+ *   path (e.g. parsers/converter) has runtime access. Keeps a single source of truth for enum
+ *   values and avoids duplicating them in test mocks.
+ */
+
+/** Whether an occurrence position holds a word or punctuation. */
+export enum OccurrenceType {
+  /** A word occurrence. */
+  Word = 'word',
+  /** A punctuation occurrence. */
+  Punctuation = 'punctuation',
+}
+
+/** The kind of linguistic analysis represented. */
+export enum AnalysisType {
+  /** Surface wordform only — no gloss or morpheme breakdown. */
+  Wordform = 'wordform',
+  /** Morpheme-level analysis with MorphemeBundles. */
+  Morph = 'morph',
+  /** Word-level gloss (no morpheme decomposition). */
+  Gloss = 'gloss',
+  /** Punctuation placeholder. */
+  Punctuation = 'punctuation',
+}
+
+/**
+ * How the analysis was produced.
+ *
+ * - `guess`
+ * - `low`
+ * - `medium`
+ * - `high`
+ */
+export enum Confidence {
+  Guess = 'guess',
+  Low = 'low',
+  Medium = 'medium',
+  High = 'high',
+}
+
+/**
+ * Lifecycle status of an assignment or alignment link.
+ *
+ * - `approved` — human-confirmed.
+ * - `suggested` — machine-generated or unreviewed.
+ * - `candidate` — proposed but not yet reviewed.
+ * - `rejected` — explicitly rejected by a human.
+ */
+export enum AssignmentStatus {
+  Approved = 'approved',
+  Suggested = 'suggested',
+  Candidate = 'candidate',
+  Rejected = 'rejected',
+}
diff --git a/src/types/interlinearizer.d.ts b/src/types/interlinearizer.d.ts
index 75736ff6..2bcd6f78 100644
--- a/src/types/interlinearizer.d.ts
+++ b/src/types/interlinearizer.d.ts
@@ -2,74 +2,483 @@
  * @file Extension type declaration file. Platform.Bible shares this with other extensions. Types
  *   exposed here (and in papi-shared-types) are available to other extensions.
  */
+
 /**
- * Interlinear types (InterlinearData, VerseData, ClusterData, etc.) are the public API for
- * interlinear data. The XML parser in src/parsers/interlinearXmlParser.ts consumes raw
- * fast-xml-parser output internally and returns objects conforming to these types.
+ * Interlinearizer Interlinear Model
+ *
+ * A representation for interlinear data that should cover import from LCM (FieldWorks), Paratext 9,
+ * and BT Extension and support the new interlinearizer
  */
 declare module 'interlinearizer' {
-  /** Character range in source text (Index, Length). */
-  export interface StringRange {
-    /** Start index of the range in the source text (0-based). */
-    Index: number;
-    /** Number of characters in the range. */
-    Length: number;
+  // ---------------------------------------------------------------------------
+  // Enums
+  // ---------------------------------------------------------------------------
+
+  /** Whether an occurrence position holds a word or punctuation. */
+  export enum OccurrenceType {
+    /** A word occurrence. */
+    Word = 'word',
+    /** A punctuation occurrence. */
+    Punctuation = 'punctuation',
+  }
+
+  /** The kind of linguistic analysis represented. */
+  export enum AnalysisType {
+    /** Surface wordform only — no gloss or morpheme breakdown. */
+    Wordform = 'wordform',
+    /** Morpheme-level analysis with MorphemeBundles. */
+    Morph = 'morph',
+    /** Word-level gloss (no morpheme decomposition). */
+    Gloss = 'gloss',
+    /** Punctuation placeholder. */
+    Punctuation = 'punctuation',
+  }
+
+  /**
+   * How the analysis was produced.
+   *
+   * - `high`
+   * - `medium`
+   * - `low`
+   * - `guess`
+   */
+  export enum Confidence {
+    Guess = 'guess',
+    Low = 'low',
+    Medium = 'medium',
+    High = 'high',
   }
 
-  /** Data on the interlinearization of a single lexeme. */
-  export interface LexemeData {
-    /** ID of the lexeme (e.g. from Lexicon; XML attribute Id). */
-    LexemeId: string;
-    /** ID of the sense/gloss used for this lexeme (XML attribute GlossId). */
-    SenseId: string;
+  /**
+   * Lifecycle status of an assignment or alignment link.
+   *
+   * - `approved` — human-confirmed.
+   * - `suggested` — machine-generated or unreviewed.
+   * - `candidate` — proposed but not yet reviewed.
+   * - `rejected` — explicitly rejected by a human.
+   */
+  export enum AssignmentStatus {
+    Approved = 'approved',
+    Suggested = 'suggested',
+    Candidate = 'candidate',
+    Rejected = 'rejected',
   }
 
-  /** Data on the interlinearization of a cluster. */
-  export interface ClusterData {
-    /** Character range this cluster occupies in the verse text. */
-    TextRange: StringRange;
-    /** Lexemes in this cluster, in order. */
-    Lexemes: LexemeData[];
-    /** Slash-joined LexemeIds for this cluster (e.g. "Word:a/Word:b"). */
-    LexemesId: string;
-    /** Unique cluster id: LexemesId plus TextRange (e.g. "Word:a/Word:b/21-3"). */
-    Id: string;
-    /** Excluded flag. See [pt9-xml.md](../parsers/pt9-xml.md) for details. */
-    Excluded: boolean;
+  // ---------------------------------------------------------------------------
+  // §1.1 Interlinearization
+  // ---------------------------------------------------------------------------
+
+  /**
+   * Top-level container for all interlinear data.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: one `IScripture` instance (singleton per project).
+   * - Paratext: merged from per-book, per-language `InterlinearData` files.
+   * - BT Extension: one `Translation` (project scope).
+   */
+  export interface Interlinearization {
+    id: string;
+
+    /** Writing system of the source text being analyzed. */
+    sourceWritingSystem: string;
+
+    /**
+     * Writing systems in which analyses are provided (e.g. `["en", "fr"]`). A single interlinear
+     * can hold analyses in multiple languages.
+     */
+    analysisLanguages: string[];
+
+    /** Books of scripture (or other texts) that have been analyzed. */
+    books: AnalyzedBook[];
   }
 
-  /** Data on punctuation change. */
-  export interface PunctuationData {
-    /** Character range this punctuation occupies in the verse text. */
-    TextRange: StringRange;
-    /** Punctuation text before the change (or empty). */
-    BeforeText: string;
-    /** Punctuation text after the change (or empty). */
-    AfterText: string;
+  // ---------------------------------------------------------------------------
+  // §1.2 AnalyzedBook
+  // ---------------------------------------------------------------------------
+
+  /**
+   * One book of scripture (or other text unit) analyzed within an Interlinear.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: `IScrBook`. `bookRef` = `BookId` (3-letter SIL code).
+   * - Paratext: book-level `InterlinearData` (merged across languages).
+   * - BT Extension: one book within a `Translation`.
+   */
+  export interface AnalyzedBook {
+    id: string;
+
+    /** Book identifier (e.g. `"GEN"`, `"MAT"`). */
+    bookRef: string;
+
+    /**
+     * Hash or version stamp of the source text at analysis time. Used to detect when the underlying
+     * text has changed and analyses may be stale.
+     */
+    textVersion: string;
+
+    /** Ordered segments that compose this book. */
+    segments: Segment[];
+  }
+
+  // ---------------------------------------------------------------------------
+  // §1.3 Segment
+  // ---------------------------------------------------------------------------
+
+  /**
+   * A sentence, clause, or verse — the unit within which occurrences are ordered.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: `ISegment` owned by `IScrTxtPara` within `IScrSection`.
+   * - Paratext: a verse (`VerseRef`) within `VerseData`.
+   * - BT Extension: a `Verse` (BCV identifier).
+   */
+  export interface Segment {
+    id: string;
+
+    /** Canonical reference (e.g. verse reference, paragraph index + offset range). */
+    segmentRef: string;
+
+    /** Raw text of the segment, for display and validation. */
+    baselineText?: string;
+
+    /** Idiomatic translation of the segment. */
+    freeTranslation?: MultiString;
+
+    /** Word-for-word translation. */
+    literalTranslation?: MultiString;
+
+    /** Ordered word / punctuation tokens in this segment. */
+    occurrences: Occurrence[];
   }
 
-  /** Interlinear data for a single verse. */
-  export interface VerseData {
-    /** Hash of verse text when approved; empty string if not approved. */
-    Hash: string;
-    /** Lexeme clusters in this verse. */
-    Clusters: ClusterData[];
-    /** Punctuation changes in this verse. */
-    Punctuations: PunctuationData[];
+  /** A string value keyed by writing-system tag. */
+  export type MultiString = Record;
+
+  // ---------------------------------------------------------------------------
+  // §1.4 Occurrence
+  // ---------------------------------------------------------------------------
+
+  /**
+   * A single word or punctuation token at a specific position in the text. Inherits its text
+   * version from the parent AnalyzedBook.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: entry in `ISegment.AnalysesRS` at a given index.
+   * - Paratext: `ClusterData` within `VerseData`.
+   * - BT Extension: `Token` (API) / `Instance` (DB).
+   */
+  export interface Occurrence {
+    id: string;
+
+    /** Parent segment. */
+    segmentId: string;
+
+    /** Zero-based position within the segment (preserves word order). */
+    index: number;
+
+    /**
+     * Positional anchor in the source text. Supports BCVWP, BCVWP+partNum, StringRange, or
+     * character offset depending on source system.
+     */
+    anchor: string;
+
+    /** The text as it appears in the source. */
+    surfaceText: string;
+
+    /** Writing system of `surfaceText`. */
+    writingSystem: string;
+
+    type: OccurrenceType;
+
+    /** All analysis assignments for this occurrence (zero or more). */
+    assignments: AnalysisAssignment[];
   }
 
-  /** Root interlinear data: book + verses. */
-  export interface InterlinearData {
-    /** Source text / project name (e.g. from InterlinearData ScrTextName attribute). */
-    ScrTextName: string;
-    /** Language code or name for the glosses. */
-    GlossLanguage: string;
-    /** Book id (e.g. "RUT", "MAT"). */
-    BookId: string;
+  // ---------------------------------------------------------------------------
+  // §1.5 Analysis
+  // ---------------------------------------------------------------------------
+
+  /**
+   * A reusable analysis describing a linguistic interpretation of a word. The same analysis can be
+   * assigned to many occurrences.
+   *
+   * Confidence and provenance belong to the analysis itself because they describe how the
+   * interpretation was produced.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: `IWfiAnalysis` (morph), `IWfiGloss` (gloss), or bare `IWfiWordform` (wordform).
+   * - Paratext: `LexemeCluster` + `WordAnalysis`.
+   * - BT Extension: synthesized from `Token.gloss` / `lemmaText` / `senseIds`. Requires deduplication
+   *   — BT Extension stores gloss/sense per-token, not as shared analysis objects.
+   */
+  export interface Analysis {
+    id: string;
+
+    /** Writing system of the analysis (e.g. the gloss language). */
+    analysisLanguage: string;
+
+    analysisType: AnalysisType;
+
+    confidence: Confidence;
+
+    /** System that produced the analysis (e.g. "lcm", "paratext"). */
+    sourceSystem: string;
+
+    /**
+     * User or automation identifier within the source system (e.g. "jsmith", "parser-v3",
+     * "auto-glosser"). Use a stable automation ID when no human directly applied the analysis.
+     */
+    sourceUser: string;
+
+    /** Word-level gloss text. */
+    glossText?: string;
+
+    /** Part of speech. */
+    pos?: string;
+
+    /** Morphosyntactic feature structure. */
+    features?: Record;
+
+    /** Ordered morpheme breakdown, when analysis is at the morpheme level (`analysisType = morph`). */
+    morphemeBundles?: MorphemeBundle[];
+  }
+
+  // ---------------------------------------------------------------------------
+  // §1.6 AnalysisAssignment
+  // ---------------------------------------------------------------------------
+
+  /**
+   * The join between an occurrence and an analysis. Multiple assignments per occurrence enable
+   * competing analyses.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: `ISegment.AnalysesRS[i]` referencing `IWfiGloss` or `IWfiAnalysis`.
+   * - Paratext: `ClusterData` with selected `LexemeData`.
+   * - BT Extension: `Token` linked to senses (`senseIds`). Status inferred from
+   *   `Instance.termStatusNum` (BiblicalTermStatus enum).
+   */
+  export interface AnalysisAssignment {
+    id: string;
+
+    /** The occurrence being analyzed. */
+    occurrenceId: string;
+
+    /** The analysis applied. */
+    analysisId: string;
+
+    /** Whether a human has confirmed this analysis for this occurrence. */
+    status: AssignmentStatus;
+
+    /** Timestamp of when the assignment was made. */
+    createdAt?: string;
+  }
+
+  // ---------------------------------------------------------------------------
+  // §1.7 MorphemeBundle
+  // ---------------------------------------------------------------------------
+
+  /**
+   * An ordered morpheme within a morpheme-level analysis, linking to the lexicon.
+   *
+   * The four optional lexicon references mirror LCM's `IWfiMorphBundle` three-way link plus the
+   * owning entry:
+   *
+   * `allomorphRef` → `IMoForm` (which surface form / allomorph) `lexemeRef` → `ILexEntry` (owning
+   * dictionary entry) `senseRef` → `ILexSense` (which meaning) `grammarRef` → `IMoMorphSynAnalysis`
+   * (grammatical behaviour)
+   *
+   * In LCM an `ILexEntry` owns one _LexemeForm_ (the elsewhere / citation allomorph) and
+   * zero-or-more _AlternateForms_ — both are `IMoForm`. `allomorphRef` identifies the specific
+   * `IMoForm` matched in this context; `lexemeRef` identifies the entry that owns it.
+   *
+   * `form` vs `allomorphRef` — `form` is the surface text of the morpheme as it appeared in this
+   * specific analysis context. `allomorphRef` is a reference (ID) to the canonical allomorph object
+   * in the lexicon. These can legitimately differ: in LCM `IWfiMorphBundle.Form` may reflect
+   * phonological conditioning that differs from the canonical `IMoForm.Form`. When `allomorphRef`
+   * is absent, `form` is the only record of the morpheme shape.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: `IWfiMorphBundle` (1:1). `allomorphRef` = GUID of `IWfiMorphBundle.MorphRA` (`IMoForm`).
+   *   `lexemeRef` = GUID of the `ILexEntry` that owns that `IMoForm` (via `LexemeFormOA` or
+   *   `AlternateFormsOS`).
+   * - Paratext: each `Lexeme` within a `WordAnalysis`. Paratext's built-in XML lexicon has no
+   *   allomorph concept distinct from the entry — `Lexeme.AlternateForms` exists in the interface
+   *   but returns empty. `allomorphRef` is therefore omitted for the built-in lexicon. When an
+   *   integrated provider (e.g. FLEx via `IntegratedLexicalProvider`) is active, `AllomorphEntry`
+   *   surfaces actual allomorphs and `allomorphRef` can be populated. `lexemeRef` = `Lexeme.Id`
+   *   (LexemeKey-derived).
+   * - BT Extension: not natively modeled as morpheme bundles. A whole-word bundle can be synthesized:
+   *   `form` = `Token.text`, `allomorphRef` = `headwordId` (the BT Extension "morph" concept
+   *   corresponds to the FieldWorks Allomorph; the HeadWord's lemma is the elsewhere / LexemeForm
+   *   allomorph), `lexemeRef` = `headwordId`, `senseRef` = `senseIds[0]`. Macula TSV `morph` field
+   *   can supply the specific allomorphic form when it differs from the lemma.
+   */
+  export interface MorphemeBundle {
+    id: string;
+
+    /** Zero-based position within the analysis (preserves morpheme order). */
+    index: number;
+
+    /** The morpheme form as it appears in this analysis (surface text). */
+    form: string;
+
+    /** Writing system of `form`. */
+    writingSystem: string;
+
+    /**
+     * Reference to a specific Allomorph (`IMoForm`) in the lexical model.
+     *
+     * An `ILexEntry` in LCM owns one _LexemeForm_ (the elsewhere / citation allomorph) and
+     * zero-or-more _AlternateForms_. This field identifies which allomorph was matched in this
+     * morpheme position.
+     *
+     * In the BT Extension the "morph" concept aligns with this field: the HeadWord's lemma acts as
+     * the LexemeForm (elsewhere allomorph).
+     */
+    allomorphRef?: string;
+
+    /** Reference to Lexeme (`ILexEntry`) in the lexical model. */
+    lexemeRef?: string;
+
+    /** Reference to Sense (`ILexSense`) in the lexical model. */
+    senseRef?: string;
+
+    /** Reference to Grammar / MSA (`IMoMorphSynAnalysis`) in the lexical model. */
+    grammarRef?: string;
+  }
+
+  // ---------------------------------------------------------------------------
+  // §1.8 InterlinearAlignment
+  // ---------------------------------------------------------------------------
+
+  /**
+   * A project pairing a source-language interlinearization and a target-language interlinear with
+   * morph-level alignment links between them.
+   *
+   * Both interlinearizations carry their own analyzed books, segments, occurrences, and analyses.
+   * AlignmentLinks bridge the two, connecting individual morphemes (MorphemeBundles) or whole
+   * unanalyzed words (Occurrences) across the language boundary.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: LCM has no native alignment or bilingual pairing model. An InterlinearAlignment is
+   *   constructed by pairing a Scripture- based interlinearization (vernacular) with a source-text
+   *   interlinearization produced externally (e.g. Greek/Hebrew resource text).
+   * - Paratext: not directly represented. Can be constructed from parallel projects that share the
+   *   same versification.
+   * - BT Extension: one `Translation` scoped to source + target sides (`Translation.sideNum`: 1 =
+   *   source, 2 = target). Each side becomes an `Interlinearization`. `Alignment` records become
+   *   `AlignmentLink`s.
+   */
+  export interface InterlinearAlignment {
+    id: string;
+
+    /** The source-language interlinearization (e.g. Greek / Hebrew). */
+    source: Interlinearization;
+
+    /** The target-language interlinearization (e.g. vernacular translation). */
+    target: Interlinearization;
+
+    /**
+     * Morph-level alignment links connecting endpoints in the source interlinear to endpoints in
+     * the target interlinear.
+     */
+    links: AlignmentLink[];
+  }
+
+  // ---------------------------------------------------------------------------
+  // §1.9 AlignmentLink
+  // ---------------------------------------------------------------------------
+
+  /**
+   * A directional alignment link from one or more source-text morphemes / words to one or more
+   * target-text morphemes / words.
+   *
+   * Each endpoint resolves to either:
+   *
+   * - A specific MorphemeBundle within a fully analyzed occurrence, connecting at the allomorph level
+   *   (via `allomorphRef`).
+   * - A whole unanalyzed occurrence, when no morpheme-level analysis exists.
+   *
+   * Typical workflow: the user selects a morph from the source-text interlinear and connects it to
+   * an allomorph of a fully analyzed occurrence in the target-text interlinear — or to an
+   * unanalyzed occurrence if the target word has not yet been broken into morphemes.
+   *
+   * Source-system mapping:
+   *
+   * - LCM: no native alignment model; links are produced by external tools.
+   * - Paratext: not stored in interlinear data; derivable from parallel interlinear selections when
+   *   two projects share versification.
+   * - BT Extension: `Alignment` entity. Each `Alignment` record with `sourceInstances` /
+   *   `targetInstances` is decomposed into `AlignmentEndpoint`s — one per instance. BT Extension's
+   *   "morph" concept (the token's morphological form) maps to a MorphemeBundle-level endpoint when
+   *   a morpheme analysis is present; otherwise the endpoint targets the whole occurrence. `status`
+   *   from `statusNum` via BT Extension's `AlignmentStatus` enum (CREATED=0, REJECTED=1,
+   *   APPROVED=2, NEEDS_REVIEW=3) — lossy mapping where both CREATED and NEEDS_REVIEW collapse to
+   *   `candidate`. `origin` from `originNum` — an undocumented integer with no enum; descriptive
+   *   strings must be defined externally. Eflomal-generated alignments leave `originNum` and
+   *   `statusNum` unset, so both default to 0 (`CREATED`).
+   */
+  export interface AlignmentLink {
+    id: string;
+
+    /** Source-side endpoints (one or more morphemes / words from the source interlinear). */
+    sourceEndpoints: AlignmentEndpoint[];
+
+    /** Target-side endpoints (one or more morphemes / words from the target interlinear). */
+    targetEndpoints: AlignmentEndpoint[];
+
+    status: AssignmentStatus;
+
+    /** How the alignment was created (manual, automatic tool, etc.). */
+    origin?: string;
+
+    /**
+     * Confidence in this alignment link, independent of the confidence on the analyses at each
+     * endpoint.
+     */
+    confidence?: Confidence;
+
+    /** Multilingual notes keyed by writing system (e.g. UI locale). */
+    notes?: MultiString;
+  }
+
+  // ---------------------------------------------------------------------------
+  // §1.10 AlignmentEndpoint
+  // ---------------------------------------------------------------------------
+
+  /**
+   * One side of an alignment link, identifying a precise point of connection within an interlinear
+   * text.
+   *
+   * When the referenced occurrence has a morpheme-level analysis, `bundleId` identifies the
+   * specific MorphemeBundle — and by extension its `allomorphRef` (IMoForm), `lexemeRef`
+   * (ILexEntry), `senseRef` (ILexSense), and `grammarRef` (IMoMorphSynAnalysis).
+   *
+   * When the occurrence is unanalyzed, `bundleId` is absent and the link targets the whole word.
+   *
+   * Resolution chain (fully analyzed): AlignmentEndpoint → Occurrence → AnalysisAssignment →
+   * Analysis → MorphemeBundle → allomorphRef (IMoForm) → lexemeRef (ILexEntry) → senseRef
+   * (ILexSense) → grammarRef (IMoMorphSynAnalysis)
+   *
+   * Resolution chain (unanalyzed): AlignmentEndpoint → Occurrence → surfaceText only
+   */
+  export interface AlignmentEndpoint {
+    /** The word or punctuation occurrence in the text. */
+    occurrenceId: string;
+
     /**
-     * Verse data keyed by verse reference (e.g. "RUT 3:1"). Exactly one entry per reference; the
-     * parser rejects XML that contains duplicate verse references.
+     * Identifies a specific MorphemeBundle within one of the occurrence's analyses. When set, the
+     * alignment connects at the allomorph / morpheme level. When absent, the alignment connects to
+     * the whole (unanalyzed) occurrence.
      */
-    Verses: Record;
+    bundleId?: string;
   }
 }
diff --git a/test-data/Interlinear_en_JHN.xml b/test-data/Interlinear_en_JHN.xml
new file mode 100644
index 00000000..ad78d4ad
--- /dev/null
+++ b/test-data/Interlinear_en_JHN.xml
@@ -0,0 +1,407 @@
+
+
+
+  
+    
+      JHN 1:1
+      
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          ,
+        
+        
+          
+          ,
+        
+        
+          
+          .
+        
+      
+    
+    
+      JHN 1:2
+      
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          .
+        
+      
+    
+    
+      JHN 1:3
+      
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          .
+        
+        
+          
+          ,
+        
+        
+          
+          .
+        
+      
+    
+    
+      JHN 1:4
+      
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          ,
+        
+        
+          
+          .
+        
+      
+    
+    
+      JHN 1:5
+      
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          
+          
+        
+        
+          
+          
+        
+        
+          
+          ,
+        
+        
+          
+          .
+        
+      
+    
+  
+
diff --git a/test-data/Interlinear_en_MAT.xml b/test-data/Interlinear_en_MAT.xml
deleted file mode 100644
index 1841e16a..00000000
--- a/test-data/Interlinear_en_MAT.xml
+++ /dev/null
@@ -1,423 +0,0 @@
-
-
-  
-    
-      MAT 1:1
-      
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:2
-      
-        
-          
-          
-          
-        
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:7
-      
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:9
-      
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:10
-      
-        
-          
-          
-        
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:11
-      
-    
-    
-      MAT 1:12
-      
-    
-    
-      MAT 1:13
-      
-    
-    
-      MAT 1:14
-      
-    
-    
-      MAT 1:15
-      
-    
-    
-      MAT 1:16
-      
-    
-    
-      MAT 1:17
-      
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:3
-      
-        
-          
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:4
-      
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-      
-    
-    
-      MAT 4:14
-      
-        
-          
-          
-        
-        
-          
-          
-        
-      
-    
-    
-      MAT 4:23
-      
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:25
-      
-        
-          
-          
-        
-      
-    
-    
-      MAT 1:5
-      
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          
-          
-        
-        
-          
-          
-        
-        
-          
-          ? 
-          ? 
-        
-      
-    
-    
-      MAT 1:0
-      
-    
-    
-      MAT 1:7kadh
-      
-    
-    
-      MAT 1:8-hEdllo
-      
-    
-    
-      MAT 1:10T
-      
-    
-    
-      MAT 1:11fff
-      
-    
-    
-      MAT 1:12qqqqqqqqqqffffffff
-      
-    
-    
-      MAT 1:13Jeff
-      
-    
-    
-      MAT 1:13JEff
-      
-    
-    
-      MAT 1:14Why
-      
-    
-    
-      MAT 1:19amwhEn
-      
-    
-    
-      MAT 1:15C2
-      
-    
-    
-      MAT 1:18
-      
-    
-    
-      MAT 1:19
-      
-    
-    
-      MAT 1:20
-      
-    
-    
-      MAT 1:21
-      
-    
-    
-      MAT 1:22
-      
-    
-    
-      MAT 1:23
-      
-    
-    
-      MAT 1:24
-      
-    
-  
-
diff --git a/test-data/Lexicon.xml b/test-data/Lexicon.xml
new file mode 100644
index 00000000..8a1663da
--- /dev/null
+++ b/test-data/Lexicon.xml
@@ -0,0 +1,390 @@
+
+
+
+  en
+  Scheherazade New
+  12
+  
+  
+    
+      
+      
+        
+          in
+        
+      
+    
+    
+      
+      
+        
+          the
+        
+      
+    
+    
+      
+      
+        
+          beginning
+        
+      
+    
+    
+      
+      
+        
+          begin
+        
+      
+    
+    
+      
+      
+        
+          -ing
+        
+      
+    
+    
+      
+      
+        
+          was
+        
+      
+    
+    
+      
+      
+        
+          Word
+        
+      
+    
+    
+      
+      
+        
+          and
+        
+      
+    
+    
+      
+      
+        
+          with
+        
+      
+    
+    
+      
+      
+        
+          God
+        
+      
+    
+    
+      
+      
+        
+          same
+        
+      
+    
+    
+      
+      
+        
+          all
+        
+      
+    
+    
+      
+      
+        
+          things
+        
+      
+    
+    
+      
+      
+        
+          thing
+        
+      
+    
+    
+      
+      
+        
+          -s
+        
+      
+    
+    
+      
+      
+        
+          were
+        
+      
+    
+    
+      
+      
+        
+          made
+        
+      
+    
+    
+      
+      
+        
+          make
+        
+      
+    
+    
+      
+      
+        
+          -ed
+        
+      
+    
+    
+      
+      
+        
+          through
+        
+      
+    
+    
+      
+      
+        
+          him
+        
+      
+    
+    
+      
+      
+        
+          without
+        
+      
+    
+    
+      
+      
+        
+          with
+        
+      
+    
+    
+      
+      
+        
+          out
+        
+      
+    
+    
+      
+      
+        
+          nothing
+        
+      
+    
+    
+      
+      
+        
+          no
+        
+      
+    
+    
+      
+      
+        
+          that
+        
+      
+    
+    
+      
+      
+        
+          has
+        
+      
+    
+    
+      
+      
+        
+          been
+        
+      
+    
+    
+      
+      
+        
+          be
+        
+      
+    
+    
+      
+      
+        
+          -en
+        
+      
+    
+    
+      
+      
+        
+          life
+        
+      
+    
+    
+      
+      
+        
+          light
+        
+      
+    
+    
+      
+      
+        
+          of
+        
+      
+    
+    
+      
+      
+        
+          men
+        
+      
+    
+    
+      
+      
+        
+          shines
+        
+      
+    
+    
+      
+      
+        
+          shine
+        
+      
+    
+    
+      
+      
+        
+          darkness
+        
+      
+    
+    
+      
+      
+        
+          dark
+        
+      
+    
+    
+      
+      
+        
+          -ness
+        
+      
+    
+    
+      
+      
+        
+          has not
+        
+      
+    
+    
+      
+      
+        
+          have
+        
+      
+    
+    
+      
+      
+        
+          not
+        
+      
+    
+    
+      
+      
+        
+          overcome
+        
+      
+    
+    
+      
+      
+        
+          over
+        
+      
+    
+    
+      
+      
+        
+          come
+        
+      
+    
+    
+      
+      
+        
+          it
+        
+      
+    
+  
+
diff --git a/tsconfig.json b/tsconfig.json
index 6a9ab667..53a54d94 100644
--- a/tsconfig.json
+++ b/tsconfig.json
@@ -57,10 +57,11 @@
     "sourceMap": true,
     // We need a baseurl for webpack's tsconfig path aliases plugin
     "baseUrl": "./",
-    /** Paths for src-rooted imports (e.g. in src/__tests__ use "parsers/..." or "main" instead of "../../parsers/..."). */
+    /** Paths for src-rooted imports (e.g. in src/__tests__ use "parsers/..." or "types/..." instead of relative paths). */
     "paths": {
       "@main": ["src/main"],
-      "parsers/*": ["src/parsers/*"]
+      "parsers/*": ["src/parsers/*"],
+      "types/*": ["src/types/*"]
     },
     "noUnusedLocals": true,
     "noUnusedParameters": true,