johnson: A multi-format dictionary UI for Emacs
***********************************************

Overview
********

johnson is a multi-format dictionary UI for Emacs.  It

provides the functionality of programs such as GoldenDict and StarDict,

allowing you to look up words across multiple dictionaries simultaneously

and view formatted definitions in a dedicated Emacs buffer.

The package is implemented entirely in Emacs Lisp with no external

dependencies, relying on Emacs 30.1's built-in sqlite support for

efficient headword indexing.  Dictionary files are parsed natively,

without calling external tools.

In version 0.4, the package supports the DSL (ABBYY Lingvo),

StarDict, MDict, and BGL (Babylon) formats, including

dictzip-compressed files (.dsl.dz and .dict.dz) and

encrypted MDict files.  It also provides a DICT protocol client

(RFC 2229) for querying remote dictionary servers.

The architecture is modular.  The core module (johnson.el) provides

the lookup engine, results buffer, navigation, history, and a

plist-based format registry (Format registration).

The sqlite index management is delegated to johnson-db.el, which

maintains one database per dictionary file in a configurable cache

directory (Cache directory).  Shared infrastructure

for dictzip decompression lives in johnson-dictzip.el.  Each

dictionary format is handled by a separate backend

module---johnson-dsl.el, johnson-stardict.el,

johnson-mdict.el, johnson-bgl.el, and

johnson-dict.el---that registers itself with the core via the

format registry.

A typical workflow consists of three steps.  First, configure

johnson-dictionary-directories to point at your dictionary files

(Dictionary discovery).  Second, run M-x johnson-index to scan

and index all discovered dictionaries (Indexing dictionaries).

Third, use M-x johnson-lookup to look up words with dynamic

completion (Looking up words).

Here is a sample configuration using elpaca and use-package:

     (use-package johnson
       :ensure (:host github :repo "benthamite/johnson")
       :custom
       (johnson-dictionary-directories '("~/dictionaries/"))
       (johnson-default-search-scope 'all)
       (johnson-dictionary-priorities '(("Oxford English-Spanish" . -10)
                                        ("Collins English-Spanish" . -5)))
       (johnson-history-max 200)
       (johnson-history-persist t)
       (johnson-display-images t)
       :bind
       ("C-c d" . johnson-lookup))

Results are displayed in a *johnson* buffer using johnson-mode, a

read-only major mode derived from special-mode.  Each dictionary that

contains a match for the queried word gets its own collapsible section,

displayed in priority order (Display ordering).  The buffer

supports Info-like navigation: press n and p to move between

sections, TAB to collapse or expand a section, RET to follow

cross-references, l and r to navigate back and forward through

lookup history, I to toggle inline images, S to search within

definitions, b to bookmark an entry, M to remove a bookmark, and

H to browse the full lookup history.

Dictionaries are organized into groups by language pair

(e.g., ``English → Spanish'').  Groups are auto-detected from dictionary

metadata but can be customized via johnson-dictionary-groups

(Search scope and groups).

The package also provides a dictionary list buffer via M-x
johnson-list-dictionaries, showing all discovered dictionaries with

their indexing status and entry counts

(Dictionary list).

User options
************

Dictionary discovery
====================

The user option johnson-dictionary-directories specifies the

directories to scan recursively for dictionary files.  When the package

discovers dictionaries, it walks each directory in this list and

identifies files using the registered format backends' detection

functions.  Files matching the _abrv naming convention (abbreviation

tables) are automatically skipped.

The default value is ("~/.local/share/dictionaries/").  You should

change this to point at wherever your dictionary files are stored.  The

value is a list of directory paths (type: (repeat directory)), so you

can specify multiple directories if your dictionaries are spread across

different locations:

     (setopt johnson-dictionary-directories
             '("~/dictionaries/"
               "/mnt/data/dictionaries/"))

Search scope and groups
=======================

The user option johnson-default-search-scope controls the default

search scope for lookups.  When set to all (the default), lookups

search across all discovered dictionaries.  When set to group, lookups

search only within the currently active dictionary group.

     (setopt johnson-default-search-scope 'group)

You can switch between these modes interactively via

johnson-select-group (Managing groups and scope), which

also lets you choose the active group.

The user option johnson-dictionary-groups lets you define custom

dictionary groups.  When this option is nil (the default), groups are

auto-detected from dictionary metadata: each dictionary is assigned to a

group based on its source and target language pair, such as ``English →

Spanish''.

Set this option when you want to create custom groupings that differ from

the language-pair default.  For instance, you could group specialized

medical dictionaries across different language pairs into a single

``Medical'' group:

     (setopt johnson-dictionary-groups
             '(("Medical" . ("Dorland's Medical"
                             "Stedman's Medical"
                             "Medical Abbreviations"))
               ("Legal" . ("Black's Law Dictionary"
                           "Legal Terms EN-ES"))))

When johnson-dictionary-groups is non-nil, only the groups listed

here are available; the auto-detected language-pair groups are not used.

Cross-reference scope
=====================

The user option johnson-ref-scope controls the scope when following

cross-reference links (via johnson-follow-ref).  When set to all

(the default), following a cross-reference performs a lookup across all

dictionaries, just like johnson-lookup.  When set to same, the

lookup is restricted to the dictionary containing the link.

     (setopt johnson-ref-scope 'same)

If the target word is not found in the originating dictionary, the

command falls back to a full lookup across all dictionaries.

Display ordering
================

The user option johnson-dictionary-priorities controls the display

order of dictionary sections in the results buffer.  It is an alist

mapping dictionary names to integer priorities.  Lower numbers display

first.  The default priority is 0, so dictionaries without an explicit

priority entry are all treated equally and appear in discovery order.

For example, to make the ``Oxford English-Spanish'' dictionary always

appear before ``Collins English-Spanish'', assign it a lower priority:

     (setopt johnson-dictionary-priorities
             '(("Oxford English-Spanish" . -10)
               ("Collins English-Spanish" . -5)))

Importing order from GoldenDict
-------------------------------

If you have a GoldenDict (or GoldenDict-ng) installation with an

established dictionary order, the command

johnson-import-goldendict-order can import it.  The command

parses the <dictionaryOrder> section of the GoldenDict

configuration file (usually ~/.goldendict/config) and matches

each dictionary name against the dictionaries discovered by Johnson.

Matching is done first by exact name, then by a normalized comparison

that strips language tags (e.g., [en-en]) and ignores case.  The

result is presented in the *johnson-reorder* buffer, where you

can review and adjust the order before saving.  Dictionaries that could

not be matched to any GoldenDict entry are appended at the end and

highlighted with the johnson-reorder-unmatched-face face.

Lookup history
==============

The user option johnson-history-max sets the maximum number of entries

retained in the lookup history.  The default value is 100.

     (setopt johnson-history-max 200)

The variable johnson-history stores the list of previously looked-up

words and serves as the history list for completing-read in

johnson-lookup (Looking up words).  You can access

previous lookups via M-p and M-n in the completion prompt.  If

savehist-mode is active, this history persists across Emacs sessions.

When the history exceeds johnson-history-max entries, the oldest

entries are discarded.

Cache directory
===============

The user option johnson-cache-directory specifies the directory where

sqlite index files are stored.  Each dictionary gets its own sqlite

database file in this directory, named by the MD5 hash of the dictionary

file's absolute path.  This hashing scheme prevents collisions when

multiple dictionaries share similar names or reside in different

directories.

The default value is ~/.cache/johnson/.  You may want to change this

if you prefer to store caches on a faster drive, in a location that is

excluded from backups, or alongside other application caches:

     (setopt johnson-cache-directory "~/.local/cache/johnson/")

The directory is created automatically if it does not exist when a

database is first opened via johnson-db-open

(Database lifecycle).

Audio playback
==============

Some dictionary entries contain embedded audio files, such as

pronunciation recordings.  When they do, the results buffer shows a

play button (▶) that plays the audio on click.  The command

johnson-play-audio-at-point does the same for the button at

point and can be called interactively.

The user option johnson-audio-player controls which program is

used for playback.  The default is auto, which tries Emacs's

built-in play-sound-file first; if that fails (e.g., because

the Emacs binary lacks sound support), it falls back to the first

available external player listed in

johnson-audio-external-players.  On macOS, afplay is

always available, so audio playback should work out of the box with no

configuration.

If you want to use a specific external player, set

johnson-audio-player to a string naming the command:

     (setopt johnson-audio-player "mpv")

The user option johnson-audio-external-players is the list of

external commands that auto mode tries, in order.  The default

list is ("afplay" "mpv" "paplay" "aplay").  You can customize

this list if your preferred player is not included or if you want to

change the priority order.

Companion zip archives
----------------------

Many DSL dictionaries bundle audio in a companion

.dsl.files.zip archive rather than storing individual files on

disk (e.g., MyDict.dsl ==> MyDict.dsl.files.zip).

When a play button is clicked and the referenced audio file does not

exist on disk, johnson automatically checks for a companion zip

archive and extracts the file on demand.  Extracted files are cached

under the resources/ subdirectory of johnson-cache-directory

so subsequent plays are instant.

This feature requires the unzip command-line utility, which

is available by default on macOS and most Linux distributions.

The command johnson-clear-resource-cache deletes all extracted

resource files.  This is useful if you want to reclaim disk space or

force re-extraction.

DICT protocol
=============

The user option johnson-dict-enabled is a master switch that

controls whether the DICT protocol client is active.  When nil

(the default), no DICT servers are contacted and no remote

dictionaries appear in the dictionary list.  Set it to t to

enable the DICT protocol backend:

     (setopt johnson-dict-enabled t)

The user option johnson-dict-servers specifies the list of DICT

servers to query.  Each element is a (HOST . PORT) pair.  The

default value is (("dict.org" . 2628)), which points to the

public dict.org server on the standard DICT port.

You can add additional servers:

     (setopt johnson-dict-servers
             '(("dict.org" . 2628)
               ("my-dict-server.local" . 2628)))

Dictionaries discovered from DICT servers appear under the ``DICT

Servers'' group in the group selector.  The DICT backend communicates

using the RFC 2229 protocol over a TCP connection.

Images
======

The user option johnson-display-images controls whether inline

images are rendered in dictionary entries.  When t (the default),

images embedded in entries are displayed inline.  When nil, images

are suppressed.

Image support works across all format backends that include images:

DSL [s] tags, and HTML <img> tags in MDict, StarDict, and

BGL entries.

     (setopt johnson-display-images nil)   ; suppress images

You can toggle image display interactively via

johnson-toggle-images (Toggling images).

Wildcard search
===============

The user option johnson-wildcard-max-results limits the number

of headwords returned by a wildcard search.  The default value is

200.  Wildcard searches can potentially match a large number of

headwords; this limit prevents excessive memory use and keeps the

completion interface responsive.

     (setopt johnson-wildcard-max-results 500)

Wildcard patterns use ? to match a single character and *

to match zero or more characters.  Wildcards are auto-detected in

johnson-lookup: if the query contains ? or *

characters, matching headwords are collected and presented for

selection rather than performing a direct lookup.

Full-text search
================

The user option johnson-fts-enabled controls whether full-text

search data is built during indexing.  The default value is nil.

When set to t before indexing, the indexer stores definition text

in a way that enables searching within dictionary entries, not just

headwords.

     (setopt johnson-fts-enabled t)

You must re-index your dictionaries after enabling this option for

the full-text data to be built.  Once enabled, the

johnson-search command (Searching definitions)

becomes available for searching within definitions.

Eldoc integration
=================

The user option johnson-eldoc-max-length sets the maximum

number of characters shown in the eldoc display for a dictionary

definition.  The default value is 80.  Longer definitions are

truncated with an ellipsis.

     (setopt johnson-eldoc-max-length 120)

To enable eldoc integration, activate johnson-eldoc-mode in

the desired buffer (Eldoc integration commands).

Scan-popup mode
===============

The user option johnson-scan-trigger controls what action

triggers popup dictionary definitions.  The possible values are:

  * selection (the default) --- show a popup when text is selected.

  * idle --- show a popup for the word at point after an idle

     delay.

  * both --- show a popup on both selection and idle.

     (setopt johnson-scan-trigger 'both)

The user option johnson-scan-idle-delay sets the number of

seconds of idle time before a popup is shown when the trigger is

idle or both.  The default value is 1.0.

     (setopt johnson-scan-idle-delay 0.5)

The user option johnson-scan-popup-duration sets the number

of seconds a popup definition remains visible.  The default value

is 5.0.

     (setopt johnson-scan-popup-duration 10.0)

The popup display uses posframe if available, otherwise falls back

to tooltip, and finally to the echo area.

To enable scan-popup mode, activate johnson-scan-mode

(Scan-popup mode commands).

Bookmarks
=========

The user option johnson-bookmarks-file specifies the file where

bookmarked dictionary entries are persisted.  The default value is

~/.cache/johnson/bookmarks.el.

     (setopt johnson-bookmarks-file "~/.local/share/johnson/bookmarks.el")

Bookmarks allow you to save dictionary entries for later reference.

Each bookmark records the headword, dictionary name, and date.

See Managing bookmarks, for the commands that add, remove, and

list bookmarks.

History buffer
==============

The user option johnson-history-file specifies the file where

lookup history is persisted across Emacs sessions.  The default

value is ~/.cache/johnson/history.el.

     (setopt johnson-history-file "~/.local/share/johnson/history.el")

The user option johnson-history-persist controls whether lookup

history is saved to disk.  When t (the default), history entries

are written to johnson-history-file and restored on startup.

When nil, history exists only for the current Emacs session.

     (setopt johnson-history-persist nil)   ; session-only history

See Browsing history, for commands that display and manage the

history buffer.

Commands
********

Transient menu
==============

The command johnson-menu opens a transient dispatch menu that

provides quick access to all major johnson commands and toggles.  It

is bound to m in johnson-mode.

The menu is organized into the following groups:

Lookup
------

l
     Look up a word (johnson-lookup).

s
     Full-text search (johnson-search).

G
     Select group (johnson-select-group).

Navigate
--------

n
     Next section (johnson-next-section).

p
     Previous section (johnson-prev-section).

j
     Jump to section (johnson-jump-to-section).

o
     Ace link (johnson-ace-link).

RET
     Follow reference (johnson-follow-ref).

History
-------

<
     Back (johnson-history-back).

>
     Forward (johnson-history-forward).

H
     History list (johnson-history-list).

C
     Clear history (johnson-history-clear).

Bookmarks
---------

b
     Add bookmark (johnson-bookmark-add).

M
     Remove bookmark (johnson-bookmark-remove).

B
     Bookmark list (johnson-bookmark-list).

Actions
-------

g
     Refresh (johnson-refresh).

w
     Copy entry (johnson-copy-entry).

W
     Copy dictionary name (johnson-copy-dictionary-name).

a
     Play audio (johnson-play-audio-at-point).

Sections
--------

TAB
     Toggle section (johnson-toggle-section).

<backtab>
     Toggle all sections (johnson-toggle-all-sections).

e
     Expand all (johnson-expand-all).

c
     Collapse all (johnson-collapse-all).

Manage
------

d
     List dictionaries (johnson-list-dictionaries).

r
     Reorder dictionaries (johnson-reorder-dictionaries).

I
     Import GoldenDict order (johnson-import-goldendict-order).

i
     Index/re-index (johnson-index).

k
     Stop indexing (johnson-stop-indexing).

X
     Clear index (johnson-clear-index).

R
     Clear resource cache (johnson-clear-resource-cache).

Q
     Close caches (johnson-close-caches).

Options
-------

Option keys use the - prefix.  Boolean options toggle between

on and off; cycle options rotate through a list of choices.

-i
     Toggle images (johnson-display-images).  When toggled, the

     results buffer is refreshed immediately to reflect the change.

-f
     Toggle full-text search (johnson-fts-enabled).

-e
     Toggle eldoc mode (johnson-eldoc-mode).

-s
     Toggle scan mode (johnson-scan-mode).

-p
     Toggle persist history (johnson-history-persist).

-c
     Cycle search scope (johnson-default-search-scope).  Rotates

     between all and group.

-r
     Cycle ref scope (johnson-ref-scope).  Rotates between

     all and same.

The transient menu requires the transient package, which is

bundled with Emacs 29 and later.  The menu is defined in

johnson-transient.el, which is loaded automatically.

Looking up words
================

The command johnson-lookup is the primary entry point for dictionary

lookups.  When called interactively, it prompts for a word with

completing-read, defaulting to the word at point.  The completion

table is backed by sqlite prefix queries across all in-scope

dictionaries, providing dynamic suggestions as you type.  Each candidate

is annotated with the number of dictionaries containing it when there

are multiple matches (e.g., (3 dicts)).

If the dictionaries have not been indexed yet, johnson-lookup prompts

you to index them first.  When indexing is already in progress, the

lookup is deferred and will resume automatically once indexing completes.

If the query contains wildcard characters (? or *),

johnson-lookup performs a wildcard search instead of a direct

lookup.  Matching headwords are collected across all in-scope

dictionaries (up to johnson-wildcard-max-results; see

Wildcard search) and presented for selection via

completing-read.  The selected headword is then looked up

normally.

After a word is selected, the command queries all dictionaries for exact

matches (case-insensitive, accent-insensitive via

johnson-db-normalize; see Headword normalization)

and displays the results in the *johnson* buffer.  The word is pushed

onto the lookup history (Lookup history) and the

navigation history (Navigation history).

The optional WORD argument allows calling johnson-lookup from Lisp

code without prompting.

The command johnson-new-search re-invokes johnson-lookup without

arguments, prompting for a new word.  It is bound to s in

johnson-mode for quick access when you want to search for a different

word while viewing results.

The command johnson-refresh re-displays the current word by

re-querying all dictionaries and rebuilding the results buffer.  This is

useful if you have re-indexed a dictionary and want to see updated

results.  It is bound to g in johnson-mode.  Unlike a new search,

refreshing does not push a new entry onto the navigation history.

Indexing dictionaries
=====================

The command johnson-index scans all configured dictionary directories,

discovers dictionary files, and indexes (or re-indexes) those whose

index is stale or missing.  Staleness is determined by comparing the

file modification time stored in the database against the actual file's

modification time (Staleness detection and reset).

Progress is displayed in a dedicated *johnson-indexing* buffer,

showing one line per dictionary with its name, progress counter, and

result (entry count or error message).

When called interactively, indexing runs asynchronously using

timer-based cooperative multitasking, so Emacs remains responsive during

the process.  In batch or noninteractive mode, indexing runs

synchronously.  If indexing is already in progress, the command signals

an error rather than starting a second concurrent run.

The optional CALLBACK argument accepts a function to be called with no

arguments once indexing completes.  This is used internally to chain

lookup after indexing, but you can also use it in your own scripts.

Dictionaries that are already up to date are skipped with an ``up to

date'' message rather than being re-indexed.

The command johnson-stop-indexing cancels the current asynchronous

indexing run.  It cancels the timer, clears the indexing queue, and

reports how many dictionaries were processed before stopping.  This is

useful if indexing is taking too long or if you want to abort and fix a

problem before continuing.

Managing groups and scope
=========================

The command johnson-select-group lets you choose the active dictionary

group by source and target language.  It prompts you in two steps:

first for the source language, then for the target language.  Each

step offers an <all> option, enabling flexible filtering.  For

example, <all> -> French selects all dictionaries with

French as the target language regardless of source, while

Spanish -> <all> selects all dictionaries with Spanish as

the source language.  It is bound to G in johnson-mode.

Selecting <all> for both source and target sets the search scope

to all, meaning lookups search across every discovered dictionary.

Any other combination sets the scope to group, restricting lookups

to matching dictionaries.  If called from the results buffer, the

current lookup is automatically refreshed to reflect the new scope.

Navigating the results buffer
=============================

The command johnson-next-section moves point to the next dictionary

section header in the results buffer.  If point is already on a header,

it moves past it to find the next one.  When there are no more sections

below, it displays ``No more sections''.  It is bound to n in

johnson-mode.

The command johnson-prev-section moves point to the previous

dictionary section header.  When there is no previous section, it

displays ``No previous section''.  It is bound to p in johnson-mode.

The command johnson-prev-section-header is an alias for

johnson-prev-section, bound to S-TAB (<backtab>) in

johnson-mode.

The command johnson-jump-to-section prompts for a dictionary

section using completion and jumps to it.  This is useful when the

results buffer contains many dictionaries and the desired section is

not visible.  It is bound to j in johnson-mode.

The command johnson-toggle-section collapses or expands the dictionary

section at point.  Collapsed sections hide their content using an

overlay with the invisible property and display a [+] indicator.

Expanding the section removes the overlay, revealing the full entry

content.  The command works whether point is on the section header or

within the section body.  It is bound to TAB in johnson-mode.

Following cross-references
==========================

The command johnson-follow-ref activates the cross-reference button at

point.  In DSL dictionaries, cross-references are created by [ref] tags

or <<...>> markers and rendered as clickable text buttons.  By

default, activating a cross-reference triggers a new johnson-lookup

for the referenced word across all loaded dictionaries.  When

johnson-ref-scope is set to same, the lookup is restricted to

the dictionary containing the link; if no match is found there, the

command falls back to a full lookup

(Cross-reference scope).

If there is no button at point, the command displays ``No cross-reference

at point''.  It is bound to RET in johnson-mode.

Navigation history
==================

The results buffer maintains a buffer-local navigation history, similar

to the history in Info mode or eww.  Each lookup pushes the word onto

the history list.  Following a cross-reference also creates a history

entry.

The command johnson-history-back navigates to the previous word in the

history, re-executing the lookup for that word.  When at the beginning

of the history, it displays ``Beginning of history''.  It is bound to l

in johnson-mode.

The command johnson-history-forward navigates to the next word in the

history (after going back).  When at the end of the history, it displays

``End of history''.  It is bound to r in johnson-mode.

Navigation history is separate from the lookup history stored in

johnson-history (Lookup history).  The lookup history

is a global list used for completing-read recall (M-p=/=M-n), while

the navigation history is buffer-local to the *johnson* buffer and

tracks the sequence of words viewed in that buffer.

When navigating back or forward, the lookup is performed with history

push suppressed (via the internal johnson--navigating-history flag),

preventing duplicate entries from cluttering the history.

Copying entries
===============

The command johnson-copy-entry copies the current dictionary section's

entry as plain text to the kill ring.  All text properties, faces, and

markup are stripped; only the raw text content is copied.

The ``current section'' is determined by point position: whichever

dictionary section the cursor is inside.  Section boundaries are

identified by the section content overlays.  If point is not within any

section (e.g., between sections or on a header line), the command

displays ``No section at point''.

It is bound to w in johnson-mode.

The command johnson-copy-dictionary-name copies the dictionary name of

the section at point to the kill ring.  This is useful for obtaining the exact

string needed by johnson-dictionary-priorities.

The command first checks for a johnson-section-header text property

(present on header lines), then falls back to the johnson-section

overlay property (present throughout the section body).  If neither is

found, it displays ``No dictionary section at point''.

It is bound to W in johnson-mode.

Browsing dictionary directory
=============================

The command johnson-browse-dictionary opens Dired on the directory

containing the dictionary file for the section at point.  This is useful

for inspecting dictionary resources, ancillary files, or the directory

structure on disk.

It is bound to D in the transient menu (johnson-menu).

Dictionary list
===============

The command johnson-list-dictionaries opens a *johnson-dictionaries*

buffer displaying all discovered dictionaries in a tabulated list.  The

buffer uses johnson-dict-list-mode, a major mode derived from

tabulated-list-mode.

The table shows six columns: Name (the dictionary's display name),

Format (e.g., ``DSL''), Languages (the source-to-target language pair),

Entries (the number of indexed headwords), Status (``Indexed'', ``Needs

reindex'', or ``Not indexed''), and Path (the abbreviated file path).  The

list is sorted by name by default and can be re-sorted by clicking on

column headers.

The command johnson-dict-list-reindex re-indexes the dictionary at

point in the tabulated list.  It deletes the existing index file, forces

a fresh index build, and refreshes the list to show the updated entry

count and status.  It is bound to i in johnson-dict-list-mode.

The command johnson-dict-list-show-details opens a

*johnson-dict-details* buffer showing detailed information about the

dictionary at point: its name, format, source and target languages,

group, priority, file path, index path, entry count, and whether the

index is stale.  It is bound to RET in johnson-dict-list-mode.

Cache management
================

The command johnson-close-caches kills all dictionary file cache

buffers and closes all open sqlite database connections.  It also resets

the internal dictionary list and indexed state, so the next lookup will

trigger a fresh discovery and staleness check.

This is useful if you have added or removed dictionary files and want to

force a clean rediscovery, or if you want to reclaim the memory used by

cached dictionary buffers.

The command reports how many cache buffers were closed and confirms that

all database connections were closed.

The command johnson-clear-resource-cache deletes the

resources/ subdirectory of johnson-cache-directory,

removing all audio files extracted from companion zip archives

(see Audio playback).

Toggling images
===============

The command johnson-toggle-images toggles the display of

inline images in the results buffer.  When images are currently

shown, the command hides them; when hidden, it re-renders the

buffer with images visible.  The toggle affects the value of

johnson-display-images (Images).

It is bound to I in johnson-mode.

Searching definitions
=====================

The command johnson-search performs a full-text search across

all indexed dictionary definitions.  It prompts for a search query

and displays results in a tabulated list showing the headword,

dictionary name, and a snippet of the matching definition with

highlighted matches.

Full-text search must be enabled via johnson-fts-enabled

(Full-text search) before indexing for this command to

produce results.  If full-text data has not been indexed, the

command signals an error.

It is bound to S in johnson-mode.

Eldoc integration
=================

The command johnson-eldoc-mode is a buffer-local minor mode

that adds dictionary definitions to eldoc.  When enabled, moving

point over a word displays a brief dictionary definition in the

eldoc area (typically the echo area or an eldoc buffer).  The

lighter is " JDict".

The maximum length of the displayed definition is controlled by

johnson-eldoc-max-length (Eldoc integration).

     ;; Enable in text-mode buffers
     (add-hook 'text-mode-hook #'johnson-eldoc-mode)

Scan-popup mode
===============

The command johnson-scan-mode is a global minor mode that

shows popup dictionary definitions when you select text or after an

idle delay.  The lighter is " JScan".

The trigger behavior, idle delay, and popup duration are controlled

by the user options johnson-scan-trigger,

johnson-scan-idle-delay, and

johnson-scan-popup-duration (Scan-popup mode).

The popup display uses posframe if the posframe package is

available, otherwise falls back to tooltip, and finally to the

echo area.

     (johnson-scan-mode 1)   ; enable globally

Managing bookmarks
==================

The command johnson-bookmark-add bookmarks the dictionary

entry at point for later reference.  The bookmark records the

headword, dictionary name, and current date.  Bookmarks are

persisted to johnson-bookmarks-file (Bookmarks).

It is bound to m in johnson-mode.

The command johnson-bookmark-remove removes the bookmark for

the dictionary entry at point.  If the entry is not bookmarked, the

command displays a message indicating so.

It is bound to M in johnson-mode.

The command johnson-bookmark-list opens a

*johnson-bookmarks* buffer displaying all bookmarked entries

in a tabulated list.  The table has three columns: Headword,

Dictionary, and Date.

In the bookmarks buffer:

  * RET re-looks up the bookmarked headword.

  * d deletes the bookmark at point.

Browsing history
================

The command johnson-history-list opens a

*johnson-history* buffer displaying the full lookup history

in a tabulated list with three columns: Headword, Time (the

timestamp of the lookup), and Results (the number of dictionary

matches).

It is bound to H in johnson-mode.

In the history buffer, pressing RET re-looks up the headword

at point.

The command johnson-history-clear clears all lookup history

after prompting for confirmation.  This removes both the in-memory

history and, if persistence is enabled, the history file on disk.

History persistence is controlled by johnson-history-persist

and johnson-history-file (History buffer).

Functions
*********

Format registration
===================

The function johnson-register-format registers a dictionary format

backend with the core.  It accepts a plist with the following keys:

  * :name --- a unique string identifying the format (e.g., "dsl").

  * :extensions --- a list of file extension strings used to filter files

     during directory scanning (e.g., ("dsl")).

  * :detect --- a function (PATH) → BOOLEAN that determines whether

     a file at PATH is a dictionary in this format.

  * :parse-metadata --- a function (PATH) → PLIST that extracts

     metadata from the dictionary file, returning at minimum :name,

     :source-lang, and :target-lang.

  * :build-index --- a function (PATH CALLBACK) → NIL that parses

     the dictionary and calls (funcall CALLBACK HEADWORD OFFSET LENGTH)

     for each entry.

  * :retrieve-entry --- a function (PATH OFFSET LENGTH) → STRING

     that retrieves the raw entry text at the given location.

  * :render-entry --- a function (RAW-TEXT) → NIL that inserts

     rendered content into the current buffer at point.

If a format with the same :name is already registered, it is replaced.

This allows format backends to re-register themselves (e.g., after

reloading the file during development).

Backend modules typically call this function inside a

with-eval-after-load form for johnson, ensuring the core is loaded

before registration occurs.  For an example, see how johnson-dsl.el

registers the DSL format at the end of the file.

Completion at point
===================

The function johnson-completion-at-point-function provides a

completion-at-point (CAPF) backend backed by the johnson dictionary

index.  It completes the word at point using the same dynamic sqlite

prefix queries as johnson-lookup (Looking up words).

To use this function, add it to completion-at-point-functions in the

major modes where you want dictionary completion:

     (add-hook 'text-mode-hook
               (lambda ()
                 (add-to-list 'completion-at-point-functions
                              #'johnson-completion-at-point-function)))

The completion respects the current search scope: if the scope is set to

group, only dictionaries in the active group are queried.

Database lifecycle
==================

The function johnson-db-open opens or creates the sqlite database for

a given dictionary file path.  It ensures the cache directory

(Cache directory) exists, creates the metadata and

entries tables if they do not already exist, creates the index on

headword_normalized if absent, and returns the database connection

object.  This is the primary entry point for obtaining a database handle.

The function johnson-db-close closes a sqlite database connection.

Pass the connection object previously returned by johnson-db-open.

In practice, the core module caches database connections in a hash table

and closes them all at once via johnson-close-caches

(Cache management).

Metadata storage
================

The metadata table stores key-value pairs describing each dictionary:

its name, format identifier, source and target languages, source file

path, file modification time, and entry count.  These metadata entries

are written during indexing and read during dictionary discovery and

staleness detection (Staleness detection and reset).

The function johnson-db-set-metadata writes a key-value pair to the

metadata table using INSERT OR REPLACE semantics, so existing keys are

updated rather than duplicated.  Both KEY and VALUE are strings.

The function johnson-db-get-metadata retrieves a single metadata value

by key, returning nil if the key is not present in the database.

The function johnson-db-get-all-metadata returns all metadata as an

alist of (KEY . VALUE) pairs.  This is useful for displaying

comprehensive dictionary details, such as in the dictionary detail

buffer opened by johnson-dict-list-show-details

(Dictionary list).

Headword normalization
======================

The function johnson-db-normalize transforms a string for

case-insensitive and accent-insensitive search.  It performs three steps

in sequence: NFKD decomposition via ucs-normalize-NFKD-string,

removal of combining diacritical marks (Unicode general category Mn),

and downcasing via downcase.

This normalization means that searching for cafe will match headwords

such as café, CAFÉ, and Café.  Both user input and headwords are

normalized with this function before storage and querying, ensuring

consistent matching behavior.

If the input is nil or an empty string, the function returns an empty

string.

Entry insertion
===============

The function johnson-db-insert-entry inserts a single headword entry

into the database.  It takes the original HEADWORD text, a BYTE-OFFSET

(or character offset, depending on the format backend), and an

ENTRY-LENGTH specifying the entry's extent in the dictionary file.  The

headword is automatically normalized via johnson-db-normalize

(Headword normalization) before insertion.

For bulk insertion during indexing, the function

johnson-db-insert-entries-batch inserts a list of entries in a single

database transaction, which is dramatically faster than individual

inserts for large dictionaries.  Each element in the ENTRIES list is a

(HEADWORD BYTE-OFFSET BYTE-LENGTH) triple.  If an error occurs during

the transaction, it is rolled back and the error is re-signaled, leaving

the database in a consistent state.

Querying entries
================

The function johnson-db-query-exact queries the database for entries

whose normalized headword exactly matches the normalized form of WORD.

It returns a list of (HEADWORD BYTE-OFFSET BYTE-LENGTH) triples,

where HEADWORD is the original non-normalized text.  Multiple triples

may be returned when a dictionary contains duplicate headwords (e.g.,

from headword alternation) or when multiple entries share the same

normalized form.  This function is used during the lookup flow in

johnson-lookup (Looking up words).

The function johnson-db-query-prefix queries the database for distinct

headwords whose normalized form begins with the normalized PREFIX.  It

uses a SQL LIKE query on the indexed headword_normalized column for

efficient prefix matching.  The optional LIMIT argument defaults to 100.

This function powers the dynamic completion table used by

johnson-lookup.

The function johnson-db-entry-count returns the total number of rows

in the entries table.  It is used for display in the dictionary list

buffer (Dictionary list) and in indexing progress messages.

Staleness detection and reset
=============================

The function johnson-db-stale-p checks whether the index for a

dictionary file is stale or does not exist.  It compares the file

modification time stored in the database metadata (under the key

"mtime") against the actual file's modification time.  If the sqlite

database file does not exist at all, the function returns non-nil.  This

function is called during the indexing flow

(Indexing dictionaries) to determine which

dictionaries need re-indexing.

The function johnson-db-reset deletes all entries from a database in

preparation for re-indexing.  It issues DELETE FROM entries but does

not drop the table or remove metadata.  After a reset, the entry count

is zero and new entries can be inserted via

johnson-db-insert-entries-batch (Entry insertion).

DSL format detection
====================

The function johnson-dsl-detect determines whether a given file path

is a DSL dictionary file.  It checks for the .dsl extension

(case-insensitively) and then verifies that the first non-BOM content in

the file starts with #, which indicates a DSL metadata header.

The detection handles all supported encodings: it reads the BOM to

determine the encoding, skips the appropriate number of BOM bytes, and

decodes the initial content to check for the # character.  For UTF-16

files, it reads enough bytes (64) to ensure it captures the first

decoded characters.

This function is registered as the :detect handler in the format

registry and is called during dictionary discovery to identify DSL files

among all files found in the configured directories.

DSL metadata parsing
====================

The function johnson-dsl-parse-metadata extracts metadata headers from

a DSL dictionary file.  It reads only the first few kilobytes of the

file (4 KB for UTF-8, 8 KB for UTF-16) since headers always appear at

the top, making this operation efficient even for large files.

The function parses three header fields: #NAME (the dictionary's

display name), #INDEX_LANGUAGE (the source language), and

#CONTENTS_LANGUAGE (the target language).  Header values are enclosed

in double quotes in the file.

It returns a plist with keys :name, :source-lang, and

:target-lang.  If a header is absent, the corresponding value defaults

to an empty string.  The core module uses these values for dictionary

display names and for auto-detecting language-pair groups

(Search scope and groups).

DSL index building
==================

The function johnson-dsl-build-index parses an entire DSL dictionary

file and calls a CALLBACK function for each headword entry found.  The

callback receives three arguments: the headword string, the character

offset of the entry body in the decoded buffer (1-based), and the

character length of the body.

The parsing proceeds in a single pass through a decoded buffer cache.

It first skips metadata header lines (beginning with #), then

identifies entries by their structure: flush-left lines are headwords,

and indented lines (beginning with a tab or spaces) are the entry body.

Multiple consecutive flush-left lines before an indented body are treated

as multiple headwords for the same entry.

Each raw headword is expanded via the internal headword expansion

pipeline, which handles three kinds of variants:

  * Split markers: colour{/}color produces two separate headwords,

     colour and color.

  * Alternation: pre{a/b}suf expands to preasuf and prebsuf.

  * Optional parts: go(es) expands to both go and goes.

Escaped characters (\[, \], \{, \}, \(, \)) are stripped

after expansion.  Each expanded variant becomes a separate row in the

database, all pointing to the same entry body.

If a headword fails to parse, it is skipped and a count of skipped

entries is logged as a warning message at the end of the indexing run.

A key design decision is the use of decoded buffers rather than raw

byte access.  Dictionary files are opened with their detected encoding

(UTF-16LE, UTF-16BE, UTF-8 with BOM, or plain UTF-8) and cached as

multibyte Emacs buffers.  The index stores character positions rather

than byte offsets, which ensures that UTF-16 encoded files work correctly

for both indexing and retrieval.

DSL entry retrieval
===================

The function johnson-dsl-retrieve-entry retrieves the raw entry body

from a DSL dictionary file.  It takes the file path, a character offset

(CHAR-OFFSET, 1-based), and a character count (NCHARS), and returns the

corresponding substring from the decoded buffer cache.

Because the index stores character positions rather than byte offsets,

this function works correctly with all supported encodings, including

UTF-16 files where byte positions would not align with character

boundaries.

The decoded buffer cache keeps dictionary files open as hidden Emacs

buffers (named " *johnson-cache: PATH") for fast repeated lookups.

All cache buffers can be killed via johnson-close-caches

(Cache management).

DSL entry rendering
===================

The function johnson-dsl-render-entry transforms raw DSL entry text

into richly formatted content in the current buffer.  It inserts the

rendered text at point using Emacs text properties and faces rather than

HTML or shr rendering.

The rendering proceeds in several steps.  First, leading indentation

(tab or spaces) is stripped from each line.  Media references

({{...}}) are removed.  Then the text is inserted into the buffer and

processed in two passes.

The first pass handles <<...>> cross-reference markers, converting

them into clickable buttons with johnson-ref-face

(Link faces).  Each button invokes johnson-lookup for the

referenced word when activated.

The second pass processes DSL tags using a stack-based parser.  The

parser scans for tag patterns matching [tag] and [/tag], maintaining

a stack of active tags.  When a closing tag is encountered, the

corresponding face or text property is applied to the enclosed region.

The supported tags include:

  * [b], [i], [u]: text styling (Text styling faces)

  * [c] and [c NAME]: color (Color faces)

  * [sup], [sub]: superscript and subscript via display properties

  * [ex]: examples (Content type faces)

  * [*]: optional/secondary text

  * [ref]: cross-reference buttons (Link faces)

  * [url]: external URL buttons

  * [lang id=N]: language identifier (stored as text property)

  * [trn], [!trn]: translation blocks (blank line separation)

  * [com]: comments

  * [m] and [m0]-[m9]: indentation levels via line-prefix and

     wrap-prefix display properties

  * [p]: abbreviation markers (rendered as-is)

  * [']: stress marks (Text styling faces)

  * [s]: media references (content is suppressed)

Faces
*****

All faces defined in johnson-dsl.el belong to the johnson-dsl-faces

customization group.  You can customize them via M-x customize-group
RET johnson-dsl-faces RET to match your preferred color theme.  The

face johnson-section-header-face is defined in johnson.el and

belongs to the johnson group.

Text styling faces
==================

The face johnson-bold-face renders text enclosed in [b]...[/b] tags.

It inherits from bold.

The face johnson-italic-face renders text enclosed in [i]...[/i]

tags.  It inherits from italic.

The face johnson-underline-face renders text enclosed in

[u]...[/u] tags.  It sets the :underline attribute to t.

The face johnson-stress-face renders stress marks enclosed in

[']...=[/']=  tags.  It inherits from bold, making accented

syllables visually prominent in pronunciation guides.

Content type faces
==================

The face johnson-example-face renders example sentences enclosed in

[ex]...[/ex] tags.  It inherits from italic with a dim gray

foreground, visually distinguishing examples from definitions.

The face johnson-optional-face renders optional or secondary text

enclosed in [*]...[/*] tags.  It uses a muted gray foreground that

adapts to the background: gray50 on light backgrounds and gray60

on dark backgrounds.

The face johnson-comment-face renders comment annotations enclosed in

[com]...[/com] tags.  It inherits from font-lock-comment-face,

giving comments a consistent appearance with the rest of Emacs.

Link faces
==========

The face johnson-ref-face renders cross-reference links enclosed in

[ref]...[/ref] tags or <<...>> markers.  It inherits from link.

Cross-references are rendered as clickable buttons that invoke

johnson-lookup for the referenced word

(Following cross-references).

The face johnson-url-face renders external URL links enclosed in

[url]...[/url] tags.  It inherits from link.  URL links are rendered

as clickable buttons that open the URL via browse-url.

Section header face
===================

The face johnson-section-header-face renders dictionary section

headers in the results buffer (the lines using Unicode box-drawing

character ━).  It inherits from bold with the :extend attribute

set to t, ensuring the face extends to the edge of the window.

This face is defined in johnson.el (for the results buffer headers)

and also in johnson-dsl.el (for DSL-specific section headers).

Color faces
===========

DSL dictionaries use [c]...[/c] and [c NAME]...[/c] tags to apply

color to text.  The module maps DSL color names (case-insensitively) to

a fixed set of customizable Emacs faces.  All color faces define

separate foreground colors for light and dark backgrounds.

The face johnson-color-default-face is used when no color name is

specified (i.e., a bare [c] tag).  It defaults to green (dark green

on light backgrounds, green3 on dark backgrounds).

The face johnson-color-green-face is used for the DSL color names

green and darkgreen.

The face johnson-color-red-face is used for red, darkred, and

crimson.

The face johnson-color-blue-face is used for blue, darkblue, and

steelblue.

The face johnson-color-gray-face is used for gray, darkgray, and

dimgray.

The face johnson-color-brown-face is used for brown and

saddlebrown.

The face johnson-color-violet-face is used for violet, purple,

and darkviolet.

The face johnson-color-orange-face is used for orange and

darkorange.

Any unrecognized color name falls back to johnson-color-default-face.

Roadmap
*******

Version 0.1 --- DSL
===================

  * johnson-dsl.el backend for .dsl files.

  * Auto-discovery and indexing of uncompressed DSL dictionaries.

  * Encoding auto-detection (UTF-8, UTF-16LE, UTF-16BE).

  * Headword alternation ({/}), optional parts ((...)), and

     multi-headword entries.

  * Case-insensitive and accent-insensitive search via NFKD

     normalization.

  * Dynamic completing-read with sqlite-backed prefix matching.

  * Full DSL markup rendering with text properties and faces.

  * Cross-reference navigation with back/forward history.

  * Dictionary groups by language pair, with configurable scope.

  * Collapsible dictionary sections in the results buffer.

  * Dictionary list buffer (tabulated-list-mode).

  * Completion-at-point (CAPF) backend.

  * Plain-text entry copying.

Version 0.2 --- StarDict and compressed DSL
===========================================

  * johnson-stardict.el backend for .ifo, .idx, .dict, and

     .syn files.

  * Dictzip random-access support for compressed .dict.dz and

     .dsl.dz files.

  * StarDict content types: plain text, HTML, Pango markup, phonetic.

  * Include table of contents at the beginning of the buffer. Integrate it with ace-link for quick navigation.

  * Allow collapsing all sections, not just individual ones (like org does).

  * Display a buffer of all indexed dictionaries and allow the user to reorder them, so that the new order can be saved as the new order in johnson-dictionary-priorities.

       * When saving, prompt the user whether they want to set the new value via defcustom, for the current session, or to set it manually in their init file; in the latter case, copy the new value to the kill ring and display instructions on how to use it.

  * Provide basic media support (audio pronunciation).

Version 0.3 --- MDict and DSL abbreviations
===========================================

  * johnson-mdict.el backend for .mdx and .mdd files.

  * Support for MDict's HTML+CSS content.

  * _abrv.dsl abbreviation table support for DSL dictionaries

     ([p] tag expansion).

Version 0.4 (current) --- BGL, DICT, and more
=============================================

  * johnson-bgl.el backend for .bgl files, with automatic

     detection via file extension and magic bytes (0x12 0x34).

  * johnson-dict.el DICT protocol client (RFC 2229) for remote

     dictionary servers.

  * Inline image support across all format backends (DSL [s] tags,

     HTML <img> tags in MDict, StarDict, and BGL).

  * Wildcard search using ? (single character) and * (multiple

     characters) in lookups.

  * Full-text search within dictionary definitions

     (johnson-search).

  * Eldoc integration via johnson-eldoc-mode.

  * Scan-popup mode via johnson-scan-mode (popup definitions on

     selection or idle).

  * Bookmarking entries for later reference.

  * Browsable history buffer with timestamps and result counts.

  * Auto-detection of non-standard 64-bit .idx entry formats in

     StarDict dictionaries whose .ifo files omit the

     idxoffsetbits=64 declaration.

  * Clearer error message when a StarDict .idx file is missing

     (incomplete dictionary distribution).

Future features
===============

The following features are planned but not yet scheduled for a specific

version:

  * EPWING format.

Indices
*******