removed code that conflicts with qlot asdf environment

[ethanxxxl]

Overview

qlot establishes an ASFD environment to find and use project-local packages. As I have been tinkering with Lem, I have also been messing with Lem's dependencies as well. The best method I have found to incorporate my changes locally is to fork/clone the code of the dependency, then in the qlfile change the dependency from git [system] [url] to local [system] [pathname].

For a while now, this has allowed me to compile Lem with my changes baked in. There has always been weird behavior when trying to add packages in after compiling, however. This is especially a problem when adding new dependencies that no extensions or subsystems depend on yet. Even if I specified a local installation in the qlfile, asdf and quicklisp would reach out and download the existing version if I tried to load it after starting lem.

The problem seems to be that qlots configuration is being overwritten at some point. I tracked down two places where this was happening:

src/lem.lisp

the function initialize-source-registry is called during end of the build. I'm not exactly sure of the details of initializing the source registry, but it had the effect of clearing the configuration done by qlot.

extensions/lisp-mode/lisp-mode.lisp

the self-connect function, called when opening a new lisp-repl, cleared the asdf:source-registry. This line was written before Lem was using qlot, I think the problem it was trying to solve is no longer relevant.

[code-contractor-app]

✅ Code Contractor Validation: PASSED

=== Contract: contract ===

✓ Code Contractor Validation Result
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

📋 Contract Source: Repository

📊 Statistics:
  Files Changed:    2
  Lines Added:      0
  Lines Deleted:    13
  Total Changed:    13
  Delete Ratio:     1.00 (100%)

Status: PASSED ✅

🤖 AI Providers:
  - codex — model: gpt-5.4

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⚠️ Violations Found (1):

[WARNING] max_delete_ratio
  Delete ratio too high: 1.00 > 0.50
  Limit: 0.50
  Actual: 1.00

📋 Contract Configuration: contract (Source: Repository)

version: 2

trigger:
  paths:
    - "extensions/**"
    - "frontends/**/*.lisp"
    - "src/**"
    - "tests/**"
    - "contrib/**"
    - "**/*.asd"
  head_branches:
    exclude:
      - 'revert-*'

validation:
  limits:
    max_total_changed_lines: 400
    max_delete_ratio: 0.5
    max_files_changed: 10
    severity: warning

  ai:
    system_prompt: |
      You are a senior Common Lisp engineer reviewing code for Lem editor.
      Lem is a text editor with multiple frontends (ncurses, SDL2, webview).
      Focus on maintainability, consistency with existing code, and Lem-specific conventions.
    rules:
      # === File Structure ===
      - name: defpackage_rule
        prompt: |
          First form must be `defpackage` or `uiop:define-package`.
          Package name should match filename (e.g., `foo.lisp` → `:lem-ext/foo` or `:lem-foo`).
          Extensions must use `lem-` prefix (e.g., `:lem-python-mode`).

      - name: file_structure_rule
        prompt: |
          File organization (top to bottom):
          1. defpackage
          2. defvar/defparameter declarations
          3. Key bindings (define-key, define-keys)
          4. Class/struct definitions
          5. Functions and commands

      # === Style ===
      - name: loop_keywords_rule
        prompt: |
          Loop keywords must use colons: `(loop :for x :in list :do ...)`
          NOT: `(loop for x in list do ...)`

      - name: naming_conventions_rule
        prompt: |
          Naming conventions:
          - Functions/variables: kebab-case (e.g., `find-buffer`)
          - Special variables: *earmuffs* (e.g., `*global-keymap*`)
          - Constants: +plus-signs+ (e.g., `+default-tab-size+`)
          - Predicates: -p suffix for functions (e.g., `buffer-modified-p`)
          - Do NOT use -p suffix for user-configurable variables

      # === Documentation ===
      - name: docstring_rule
        prompt: |
          Required docstrings for:
          - Exported functions, methods, classes
          - `define-command` (explain what the command does)
          - Generic functions (`:documentation` option)
          Important functions should explain "why", not just "what".
        severity: warning

      # === Lem-Specific ===
      - name: internal_symbol_rule
        prompt: |
          Use exported symbols from `lem` or `lem-core` package.
          Avoid `lem::internal-symbol` access.
          If internal access is necessary, document why.

      - name: error_handling_rule
        prompt: |
          - `error`: Internal/programming errors
          - `editor-error`: User-facing errors (displayed in echo area)
          Always use `editor-error` for messages shown to users.

      - name: frontend_interface_rule
        prompt: |
          Frontend-specific code must use `lem-if:*` protocol.
          Do not call frontend implementation directly from core.
        severity: warning

      # === Functional Style ===
      - name: functional_style_rule
        prompt: |
          Prefer explicit function arguments over dynamic variables.
          Avoid using `defvar` for state passed between functions.
          Exception: Well-documented cases like `*current-buffer*`.

      - name: dynamic_symbol_call_rule
        prompt: |
          Avoid `uiop:symbol-call`. Rethink architecture instead.
          If unavoidable, document the reason.

      # === Libraries ===
      - name: alexandria_usage_rule
        prompt: |
          Alexandria utilities allowed: `if-let`, `when-let`, `with-gensyms`, etc.
          Avoid: `alexandria:curry` (use explicit lambdas)
          Avoid: `alexandria-2:*` functions not yet used in codebase

      # === Macros ===
      - name: macro_style_rule
        prompt: |
          Keep macros small. For complex logic, use `call-with-*` pattern:
          ```lisp
          (defmacro with-foo (() &body body)
            `(call-with-foo (lambda () ,@body)))
          ```
          Prefer `list` over backquote outside macros.

📚 About Code Contractor

Declarative Code Standards That Learn and Improve

Define domain-specific validation rules in YAML.
Your contracts document team knowledge and evolve into more accurate AI enforcement.

Want this for your repo?
→ Install Code Contractor

[code-contractor-app]

Code Contractor validation passed ✅ — see the sticky comment for full results.

[ethanxxxl]

@vindarel can we review and merge these changes?

[vindarel]

disclaimer: I am barely a Qlot user. Questions:

• does that change anything in how you start Lem / do you need to update the README / documentation?
• cutting the grass under asdf seems necessary (I've been caught by that kind of stuff). Can we however write out a test plan and do it after merging?

(thanks!)

[theangelperalta]

Reviewed the change and tested the ASDF behavior directly (SBCL 2.5.10). TL;DR: the diagnosis is right, the fix is sound, and removing the clear-source-registry call does not regress anything.

The key thing I wanted to confirm was the worry that deleting (asdf:clear-source-registry) from self-connect would stop the self REPL from discovering systems added after startup — since the old XXX comment claimed that's what it was for. So I set up a tree-based source registry (the Qlot-style setup), warmed the cache, dropped in a brand-new .asd, and tried each refresh path:

Action	New .asd (added at runtime) discovered?
nothing (warm cache)	no
asdf:clear-source-registry	no
asdf:clear-configuration	no
asdf:initialize-source-registry (re-supply parameter)	yes

Two takeaways:

1. clear-source-registry was not doing what its comment claimed. It clears the cache but the next lazy lookup just rebuilds the same directory listing — it does not rescan disk for new .asd files. Only a fresh initialize-source-registry forces a re-scan. So the deleted line never actually made "systems added after lem initialization" visible; removing it loses no real behavior.

2. clear-source-registry preserves the existing programmatic config (a previously-registered system still resolved after clearing — it reverts to the last initialize-source-registry parameter, i.e. Qlot's). So that call wasn't the thing clobbering Qlot's setup.

The real culprit is the src/lem.lisp initialize-source-registry, which replaced Qlot's parameter with one that :also-excluded .qlot and re-rooted the tree — that's what hid the local-pinned deps. Removing it is the substantive, correct fix.

On @vindarel's questions:

• No change to how Lem starts; no doc change strictly required (though documenting the local qlfile workflow for hacking on deps would be a nice follow-up).
• A merge-then-test plan is fine. Suggested manual check: local-pin a dependency in the qlfile, build, and confirm your changes are baked in and resolved from the local path. The "drop in a brand-new system from the running self REPL" case is worth noting as already unsupported both before and after this PR — picking up a genuinely new system at runtime still requires (asdf:initialize-source-registry …) / (ql:register-local-projects). If desired, exposing that as a small command would be a reasonable separate enhancement, not a blocker here.

LGTM.

[theangelperalta]

Opened #2203 to track the follow-up: an explicit lisp-mode command to register/refresh newly-added ASDF systems at runtime (the working replacement for what the deleted clear-source-registry was meant to do). Not a blocker for merging this PR.

[vindarel]

Let's have 3 pairs of eyes (or more) on this now ;)