Skip to content

Drive developer guide LanguageTool count to zero and gate CI on it#5007

Merged
shai-almog merged 2 commits into
masterfrom
fix-developer-guide-grammar
May 22, 2026
Merged

Drive developer guide LanguageTool count to zero and gate CI on it#5007
shai-almog merged 2 commits into
masterfrom
fix-developer-guide-grammar

Conversation

@shai-almog
Copy link
Copy Markdown
Collaborator

Summary

  • Brings the developer-guide LanguageTool job from 7613 matches → 0 by fixing real source typos / broken AsciiDoc syntax and by tightening the HTML→text extraction so the tool stops flagging code-rendering artifacts.
  • Promotes the LanguageTool step in .github/workflows/developer-guide-docs.yml from advisory to a hard quality gate — a non-zero match count now fails the build, so regressions don't pile up.
  • Adds a regex-based accept-list at docs/developer-guide/languagetool-accept.txt (Codename One product/API names, file extensions, British/American spelling pairs) as the documented escape hatch for terms the default English dictionary doesn't know.

What changed

Script (scripts/developer-guide/run_languagetool.py):

  • Fixed off-by-chunk offset bug — language_tool_python.Match.__setattr__ silently drops unknown keys, so the old _global_offset was never persisted past chunk 0.
  • Replaced stripped inline <code>/<kbd>/... with a single X placeholder so word boundaries survive (kills ~2400 CONSECUTIVE_SPACES false positives) and dropped preceding a/an articles to avoid EN_A_VS_AN mismatches.
  • Skip whole-element contents for the auto-generated table of contents, <dt>/<th>/<td>, <div class="listingblock">, headings, and the footer.
  • Accept dotted (android.permission) and underscored (android.cusom_layout1) identifiers when they appear in build-hint tables.
  • Curated DISABLED_RULES set for rules whose remaining matches are stylistic (e.g. COMMA_COMPOUND_SENTENCE, MISSING_HYPHEN, ID_CASING, EN_UNPAIRED_BRACKETS, MAC_OS).

Source corrections across ~40 .adoc/.asciidoc files, including:

  • Spelling typos: modiymodify, libarylibrary, precendenceprecedence, CodenmaeCodename, LighthoustLighthouse, ~40 distinct items
  • Brand normalization: JavascriptJavaScript, GithubGitHub, Mac OS XmacOS, Google mapsGoogle Maps
  • 73 you've a/the/...you have a/the/... (American style); 29 thruthrough
  • 200+ . it's/. for example/etc. → . It's/. For example (capitalized after period)
  • Broken AsciiDoc syntax: 118 For example, [source,X] blocks split off the prose line onto their own line
  • Italicized filenames (_InfoPlist.strings_) and link-text-of-filenames (androidTheme.res) reformatted as backticked code so grammar checks don't bleed across the dot
  • Compound-noun hyphenation: bare bonesbare-bones, pseudo codepseudo-code, cross platformcross-platform, 1 month subscription1-month subscription

Workflow (.github/workflows/developer-guide-docs.yml):

  • Step renamed from Run LanguageTool grammar check (advisory)Run LanguageTool grammar check; build-fail message points contributors at the accept list and the disable-list.
  • Quality-gate step now fails when LANGUAGETOOL_COUNT > 0 (alongside Vale, Asciidoctor, and unused-image checks).

Test plan

  • Local asciidoctor render → python3 scripts/developer-guide/run_languagetool.py reports 0 matches (verified end-to-end after the final fix).
  • CI re-runs the full quality-gate pipeline on this PR; expect Vale, Asciidoctor, paragraph-capitalization, and LanguageTool all to pass.
  • Sanity-check the rendered HTML for the five files with the highest churn (Advanced-Topics-Under-The-Hood, The-Components-Of-Codename-One, Index, css, io) — diffs are large in volume but each hunk is a small targeted prose/typography fix.
  • If a future PR introduces a legitimate identifier the dictionary doesn't know, the failure message instructs the contributor to add it to docs/developer-guide/languagetool-accept.txt.

🤖 Generated with Claude Code

Fixes proper source typos, broken AsciiDoc syntax, and HTML extraction
artifacts so the developer-guide LanguageTool job reports 0 matches
(down from 7613). Promotes that job from advisory to a hard quality
gate, so future regressions block the build instead of accumulating.

Highlights:
- scripts/developer-guide/run_languagetool.py: fix off-by-chunk
  offset bug (language_tool_python silently drops custom attributes),
  replace inline code spans with a placeholder, suppress TOC / table
  cells / heading text from grammar checking, treat dotted and
  underscored tokens as code identifiers, and load a regex-based
  accept-list per docs/developer-guide/languagetool-accept.txt.
- docs/developer-guide/languagetool-accept.txt: new file listing
  Codename One product names, API identifiers, file extensions, and
  British/American spelling pairs that the default English dictionary
  doesn't recognize.
- docs/developer-guide/*.adoc/*.asciidoc: fix real typos
  (modiy/libary/precendence/Lighthoust/...), normalize brand names
  (Javascript -> JavaScript, Github -> GitHub, Mac OS -> macOS, Google
  maps -> Google Maps), expand "you've a" -> "you have a", capitalize
  lowercase sentence starts after periods, fix 118 broken
  "For example, [source,X]" blocks that wandered onto prose lines,
  reformat italicized filenames as backticked code, and apply the
  long tail of single-occurrence fixes.
- .github/workflows/developer-guide-docs.yml: stop labelling the
  LanguageTool step "advisory" and add a LANGUAGETOOL_STATUS / COUNT
  check to the final quality-gate so a non-zero match count fails the
  build.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Comment thread scripts/developer-guide/run_languagetool.py Fixed
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented May 22, 2026

Developer Guide build artifacts are available for download from this workflow run:

Developer Guide quality checks:

  • AsciiDoc linter: No issues found (report)
  • Vale: No alerts found (report)
  • Paragraph capitalization: No paragraph capitalization issues (report)
  • LanguageTool: No grammar matches (report)
  • Image references: No unused images detected (report)

@github-actions
Copy link
Copy Markdown
Contributor

Cloudflare Preview

CI surfaced two follow-ups on the original change:

1. Vale reported 5 errors and 7 warnings whose suggestions conflict
   with what LanguageTool requires. Per the user's policy of trusting
   the grammar checker over Vale in conflicts, this commit:
   - Adds `// vale-skip:` exceptions (with reasons) for the cases
     where LanguageTool's correction is the right one: `i.e.,`
     punctuation, `it is` parsing as "of-it is-simple" (not the
     possessive contraction Vale wants), conjunctive `So` opening a
     sentence, the literal CSS Grid keyword `auto-fit`, and the
     adverb `properly` that carries real meaning.
   - Rephrases the worked-example "let's"/"Let's" openings to use
     `Suppose`/`to ... change ...`/`Now look`/`Start` instead. Both
     LanguageTool's LETS_LET rule (requires `let's`) and Vale's
     Microsoft.We rule (forbids first-person plural) are satisfied
     by removing the construct rather than excepting either side.

2. github-code-quality bot flagged the unused module-level
   `_ACCEPT_PATTERN = None` placeholder. The actual accept-list state
   flows through function parameters, so the variable was dead code;
   removed along with its now-misleading comment.

Verified locally: vale reports 0 errors/0 warnings/0 suggestions
across all 65 files and LanguageTool reports 0 matches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@shai-almog shai-almog merged commit dcd8831 into master May 22, 2026
9 checks passed
shai-almog added a commit that referenced this pull request May 22, 2026
After merging master (which promoted LanguageTool to a hard quality
gate via #5007) the developer guide had 7 grammar-checker matches
attributable to this branch's chapter plus 1 pre-existing match
elsewhere. This drives the count back to 0.

Chapter-local fixes (docs/developer-guide/On-Device-Debugging.asciidoc):
  - Replace "source dir" with "source directory" so LanguageTool's
    morfologik English dictionary stops flagging the abbreviation.
  - Rephrase the "Watching expressions follow the same rule" sentence
    so QUESTION_MARK no longer fires (LanguageTool reads the "ing"
    opening as a rhetorical fragment expecting a question mark).
  - Rephrase the "What doesn't work:" lead so the wh-word doesn't
    trip QUESTION_MARK on the surrounding sentence.

Accept-list additions (docs/developer-guide/languagetool-accept.txt):
  - jdb (the JDK command-line debugger),
  - loopback (the networking term),
  - rethrow / rethrows (standard exception-handling vocabulary).
  These are well-known technical terms LanguageTool's default
  dictionary doesn't recognise but appear unaltered in any
  reasonable developer-facing prose.

Pre-existing match (docs/developer-guide/Maven-Getting-Started.adoc):
  Replace "(aka X. aka Y)" with "(also known as X or Y)" — the
  period between the two "aka"s made LanguageTool flag the second
  "aka" as a sentence-start lowercase letter. Master's docs job is
  PR-trigger-only so this never showed up there; it surfaced here
  because the merge brought in the gate.
shai-almog added a commit that referenced this pull request May 23, 2026
* Add iOS on-device debugging support

Adds a JDWP-compatible debugger for ParparVM-built iOS apps so jdb /
IntelliJ / VS Code can attach to a real device or the iOS Simulator
and set breakpoints, walk the stack, and inspect locals + Strings.

Three pieces:
- ParparVM translator emits per-method side-tables (locals addresses,
  variable names, line tables) and a cn1-symbols.txt sidecar when
  -Dcn1.onDeviceDebug=true is set. Release builds are unaffected.
- A listener thread (Ports/iOSPort/nativeSources/cn1_debugger.{h,m})
  is compiled into debug builds, dials out to a desktop proxy over
  TCP, and services set/clear-bp, resume, step, get-stack/locals,
  get-object-class, and get-string commands. The hot path in
  __CN1_DEBUG_INFO is one predictable load+branch when nothing is
  attached.
- A new Maven module (cn1-debug-proxy) bridges that custom protocol
  to JDWP so any standard Java debugger speaks to it. Includes a
  minimum-viable JDWP implementation covering everything jdb needs
  for breakpoint, where, locals, and String inspection.

Maven goals: cn1:ios-on-device-debugging (launches the proxy) and
cn1:buildIosOnDeviceDebug (cloud build target).

Build-hint UX: codename1.arg.ios.onDeviceDebug=true plus
proxyHost/proxyPort. End-user docs live in
docs/developer-guide/On-Device-Debugging.asciidoc.

* Address review feedback on on-device debugging

- Force-off ios.onDeviceDebug on release builds (ios.buildType=release)
  in both the translator JVM flag and the Info.plist injection, so a
  stray hint in codenameone_settings.properties can't leak the debug
  listener thread into an App Store binary.
- Document the new hints (ios.onDeviceDebug, .proxyHost, .proxyPort,
  .waitForAttach) in the iOS build hints table in
  Advanced-Topics-Under-The-Hood.asciidoc.
- Drop unused Parser.getClasses() that triggered MS_EXPOSE_REP.
- Rework the dev-guide chapter: remove the {cn1-release-version}
  sentence from Prerequisites, drop the "macOS with Xcode required"
  claim (the cloud build path works equally), drop the redundant
  JDWP-debugger line, collapse the duplicated build instructions
  into one step that points at the normal build flow, switch to
  build-hint vocabulary, and strip the codename1.arg. prefixes from
  the user-facing hint names.
- Fix Vale prose-linter regressions (contractions, first-person,
  Latinisms).

* Avoid proselint Diacritical false positive on 'Resume'

* On-device debug UX: waiting overlay, non-blocking start, JDWP reattach

Quality-of-life improvements that emerged while running the proxy end-to-end
locally against the iOS simulator.

Device-side runtime (cn1_debugger.m + .h):
  - cn1_debugger_start() no longer blocks the AppDelegate on
    didFinishLaunchingWithOptions. The proxy connection runs on its own
    thread regardless of CN1ProxyWaitForAttach, so UIKit can finish boot,
    draw the launch transition, and -- when waitForAttach is on -- present
    a translucent "Waiting for debugger..." overlay UIWindow. The previous
    behaviour left the user staring at the splash with no signal that the
    app was waiting on anything.
  - New cn1_debugger_run_when_ready(block) API lets the AppDelegate defer
    the VM start callback until the proxy reports the IDE has attached.
    When waitForAttach is off (or on-device-debug is disabled at build
    time) the block runs synchronously and behaves identically to the
    pre-change boot flow.

GLAppDelegate:
  - Calls cn1_debugger_run_when_ready around the VM callback so wait-mode
    no longer races against splash dismissal, and captures the location
    launch option into the block so it survives the deferral.

JDWP proxy (JdwpServer.java):
  - acceptAndServe() now loops on accept() so the developer can detach and
    reattach the IDE without restarting the proxy. Per-attach state is
    reset via closeJdwpSession(); breakpoint registrations persist across
    attaches.
  - After handshake completes the proxy schedules an auto-resume that
    releases the device-side waitForAttach gate 500 ms later. The delay
    gives IntelliJ / VS Code time to register breakpoints before the app
    races past them; without this the app sat on the waiting overlay
    forever because most JDWP debuggers don't auto-send VM.Resume.

Misc:
  - Add /artifacts/ to .gitignore (build wrapper drop-zone used by the
    new ios-on-device-debugging mojo).

* On-device-debug: object inspection + native stdout + CN1-core BP docs

Three additions that turn the proxy from "stack-trace viewer" into a
real interactive debugger.

1. Instance-field inspection
   Translator: ByteCodeClass.appendOnDeviceDebugFieldTable emits a
   per-class field-offset table (fieldId, offsetof, JVM type char,
   name) in each generated .m file, behind a CN1_ON_DEVICE_DEBUG guard.
   An __attribute__((constructor)) shim registers the table with
   cn1_debugger at process load. Parser.writeSymbolSidecar also emits
   `field <classId> <fieldId> <name> <descriptor> <accessFlags>` rows
   so the proxy can answer JDWP ClassType.Fields / FieldsWithGeneric
   without a device round-trip.
   Device: new CMD_GET_OBJECT_FIELDS handler walks the registered
   field table for the object's runtime classId and reads each field
   straight out of the struct using offsetof. Replies as
   EVT_OBJECT_FIELDS (count, then [type-char, value] tuples).
   Proxy: JdwpServer.handleObject case 2 (GetValues), handleStackFrame
   case 3 (ThisObject), and ClassType cases 4/14 (Fields,
   FieldsWithGeneric) now read real data instead of returning empty
   stubs. ThisObject piggybacks on the existing GetLocals path,
   reading slot 0 as an object reference for instance methods (the
   JVM always parks `this` there on entry); for statics slot 0 is
   zero, which is the correct JDWP reply.
   `dump this` in jdb against a running ParparVM-built iOS app now
   shows `tickCount: 4, pointerCount: 0` etc. — actual field values.

2. Native stdout / stderr forwarding
   cn1_debugger_start dup2()'s STDOUT_FILENO and STDERR_FILENO to
   pipes, then runs two streamCaptureThreads that chunk the pipes by
   newline. Each completed line is mirrored back to the original FD
   (so xcrun simctl log / Xcode console still works) and, if a proxy
   is connected, sent as EVT_STDOUT_LINE / EVT_STDERR_LINE.
   Proxy: handles the events and prints them prefixed with `[device]`
   to its own stdout (IntelliJ surfaces this in the proxy's debug
   console when it's launched as an IDE run config).
   System.out.println, Log.p, printf, fprintf(stderr,...) all flow
   through. NSLog on the iOS Simulator works too; on a real device
   NSLog may bypass stderr (documented as a limit).

3. CN1 core class breakpoints
   Confirmed working — sidecar already covers framework classes the
   same way it covers user code. The missing piece was just docs:
   On-Device-Debugging.asciidoc now describes how to attach the
   CodenameOne/src source directory in IntelliJ / jdb so the source
   pane resolves while stepping through framework code. Also
   tightened the "What works today" list and added a fresh "Known
   limitations" entry for static-field reads, plus a note that NSLog
   on a real device may bypass the stdout forwarder.

Wire protocol additions:
  CMD_GET_OBJECT_FIELDS (0x0D), EVT_OBJECT_FIELDS (0x8A),
  EVT_STDOUT_LINE (0x8B), EVT_STDERR_LINE (0x8C).
SymbolTable additions:
  FieldInfo, ClassInfo.instanceFields, fieldById, fieldCount.

Verified end-to-end on iPhone 17 Pro / iOS 26.3 simulator under
Xcode 26 against a minimal CN1 app: BP in framework's
Display.edtLoopImpl fires, list shows the actual source, locals
populate with their real names, and `dump this` walks the
instance-field table.

* Fix EventRequest.Set walking past payload + IntelliJ console docs

Real bug found while testing stepping in IntelliJ: jdb's `next`
(step-over) returned `JDWP Error: 103` because the EventRequest.Set
modifier-loop walked past the end of the payload when JDI sent its
default ClassExclude modifiers (`java.*`, `javax.*`, `sun.*`,
`com.sun.*`, `jdk.internal.*`, etc. — JDI auto-attaches these to
every step request) and our switch's `default` branch set
`off = p.length`, then the next iteration read `p[p.length]` and
ArrayIndexOutOfBounds'd. The IDE's view: step request rejected → no
step event → app keeps running → looks indistinguishable from
"continue".

Fix: every modifier-case now bounds-checks before reading and bails
the loop via a `badModifier` flag if anything's off. Added the
missing modifier kinds JDI emits but we hadn't seen on the wire
yet (`FieldOnly`=9, `SourceNameMatch`=12), and changed the unknown-
kind branch to abort the loop with a `[jdwp]` log instead of trying
to guess the width.

Also fixed a related NPE: when the IDE detached mid-session, the
device-disconnect path tried to send VM_DEATH on an already-null
out stream and crashed the listener thread. writeEventCommand now
no-ops when out is null.

Added `[jdwp] STEP request` / `STEP_COMPLETE` / `VM.Resume` /
`Thread.Resume` log lines so future debugging of stepping is just
a matter of reading the proxy console.

Docs: documented the IntelliJ run-config trick for surfacing
device output ([device] lines) in the IDE console — launch the
proxy as an Application configuration alongside the Remote JVM
Debug attach and group them with a Compound run config. Without
this, the proxy's stdout (where device prints end up) only shows
up if the user runs the proxy from a terminal.

Verified end-to-end on iPhone 17 Pro / iOS 26.3 sim:
  - jdb's `next` from a BP at heartbeat line 41 lands on line 42
    (STEP_COMPLETE logged).
  - jdb's `step` from a BP at line 42 enters the Log.p method
    (STEP_COMPLETE for methodId=13294, line=242).
  - Proxy survives an IDE detach and accepts the next reattach.

* On-device debug: method invocation (Class/ObjectReference InvokeMethod)

The IDE can now call any framework / user method on a paused VM and
have the result come back as a real object reference (or a value, or
a thrown Throwable). This is what makes `print
Display.getInstance().getCurrent().getTitle()` work in jdb, and
what makes IntelliJ's "Evaluate Expression" pop-up usable for chains
that involve method calls.

Translator (BytecodeMethod / ByteCodeClass / Parser):
  - Emits one C "invoke thunk" per non-eliminated method, per class,
    under CN1_ON_DEVICE_DEBUG. The thunk has a uniform signature
    (tsd, this, args, *result), unpacks the args from a union into
    the typed C parameters the translated method expects, dispatches
    through virtual_<sym>() / <sym>() depending on virtuality, and
    packs the return value back into the result union. The call is
    wrapped in a catch-all try block so an uncaught Throwable
    round-trips as result.type='X' instead of longjmp-ing past
    suspendCurrent's cond_wait.
  - Skips classes whose hand-written native impl has fallen out of
    sync with the translator's calling convention: java.io.*,
    java.net.*, java.nio.*, com.codename1.impl.*. The other system
    packages (java.lang, java.util, ...) are fine because their
    native impls are in nativeMethods.m and use the modern names.
  - When on-device-debug is on, the unused-method optimiser keeps
    every instance method of java.lang.Object alive — jdb's `print`
    formats every object through Object.toString, so silently
    dropping it earlier made every evaluation return "<void value>".
  - Sidecar `method` rows now carry an isStatic flag; `class` rows
    carry their superclass id so the proxy can answer
    ClassType.Superclass and let JDI walk to inherited methods.

Device runtime (cn1_debugger.h/m):
  - New cn1_invoke_arg union + cn1_invoke_result struct (JVM type-
    char plus value slot), and a cn1_invoke_thunk_t function-pointer
    type that the translator-emitted thunks all match.
  - cn1_debugger_register_invoke_thunk(methodId, thunk) registry,
    array-indexed by methodId for O(1) lookup.
  - New CMD_INVOKE_METHOD handler. It queues the call on the
    target thread's sus_state, signals s->cv, and blocks the
    listener thread on a result-ready predicate. suspendCurrent's
    cond_wait loop services the request on the suspended Java
    thread (so the call runs in a valid tsd / GC context), then
    goes back to waiting.
  - New EVT_INVOKE_RESULT response carrying (type-char, 8-byte value).

Proxy (WireProtocol / DeviceConnection / SymbolTable / JdwpServer):
  - WireProtocol: CMD_INVOKE_METHOD=0x0E, EVT_INVOKE_RESULT=0x8D.
  - DeviceConnection: invokeMethod(threadId, methodId, thisObj,
    argTypes[], argValues[]) and onInvokeResult(type, value)
    callback wired through the listener.
  - SymbolTable: MethodInfo.isStatic, ClassInfo.superId, and
    extended `method`/`class` row parsing (still tolerates older
    4-column rows).
  - JdwpServer:
      * ClassType.InvokeMethod (cmd 3) and
        ObjectReference.InvokeMethod (cmd 6) parse the JDWP args,
        forward as a CMD_INVOKE_METHOD, and pack the device's
        EVT_INVOKE_RESULT into a JDWP returnValue + exception slot.
      * ClassType.Superclass now returns the actual sidecar superId
        so JDI walks the hierarchy properly instead of stopping at
        every class.
      * Methods / MethodsWithGeneric set the JDWP STATIC bit when
        the sidecar marks the method static — without it jdb's
        expression parser refuses to resolve `Class.method()`.

Verified end-to-end on iPhone 17 Pro / iOS 26.3 simulator. Single-
invoke and chained:

  print com.codename1.ui.Display.getInstance()              -> Display ref
  print Display.getInstance().getCurrent()                  -> Form ref
  print Display.getInstance().getCurrent().getTitle()       -> String "hello, world"

Object.toString round-trips through nativeMethods.m, so jdb's
default object-display formatting also works.

* Proxy: --trace-jdwp flag for diagnosing IDE-specific step / invoke issues

When IntelliJ's step behaviour diverges from jdb's (different default
class-exclude modifiers, additional pre-step queries, etc.), the proxy
needs a way to surface every JDWP command it receives so the wire-
level difference is visible without rebuilding. Add a --trace-jdwp
flag that toggles a single-line log per inbound command (cmd-set /
cmd / id / payload length).

Off by default — release sessions shouldn't pay the log overhead.

* Proxy: fix off-by-one in JDWP EventRequest modifier-kind numbering

JDWP modifier-kind values per the spec:
  1 Count                 2 ConditionalExpr (deprecated)
  3 ThreadOnly            4 ClassOnly
  5 ClassMatch            6 ClassExclude
  7 LocationOnly          8 ExceptionOnly
  9 FieldOnly            10 Step
 11 InstanceOnly         12 SourceNameMatch

The previous switch had every kind shifted by one — 2 was treated as
ThreadOnly (really Conditional, deprecated), 3 as ClassOnly (really
ThreadOnly), and the string-payload kinds were 4/5 instead of 5/6.
The practical bite: IntelliJ auto-attaches a handful of ClassExclude
modifiers (`java.*`, `javax.*`, `sun.*`, `com.sun.*`, `jdk.internal.*`)
to every StepRequest. With kind=6 unrecognised the parser bailed
mid-payload, sent the modKind=6 warning to the proxy log, and
truncated the modifier list — IntelliJ then either retries the step
or shows "Source code does not match the bytecode" depending on
exactly which modifiers it expected to be honoured.

Caught from a JDWP packet trace where IntelliJ sent a 561-byte
EventRequest.Set with 27 modifiers; the proxy logged
"unknown modKind=6 — ignoring remaining 26 modifiers" right before
each step.

Also added an explicit case for modKind=2 (Conditional, deprecated)
even though no current debugger sends it, so the parser handles
every value in the spec without falling to the default branch.

* Proxy: invalidate stack / locals cache on suspend & resume events

Frames panel was sticking to the previous suspend location after each
step because fetchStackForThread short-circuits on a populated cache.
onBreakpointHit eagerly fetches the stack and populates that cache;
onStepComplete had no eager fetch but also didn't invalidate, so when
IntelliJ asked for Frames after a step it got back the BP_HIT stack —
e.g. user sets BP on line 48, presses F8 once, the cursor / Frames /
double-click all still point to line 48 while the device is actually
suspended on line 49.

Same bug affected locals (StackFrame.GetValues short-circuits via
its own pendingLocals cache, kept across suspensions of the same
frameIdx) — the IDE evaluated expressions against a stale frame.

Fix: introduce invalidateStack() / invalidateLocals() helpers and
call them in every state-changing handler:
  - onBreakpointHit  (before the eager fetch)
  - onStepComplete   (no eager fetch — IDE will pull)
  - VM.Resume        (everything cached is now meaningless)
  - Thread.Resume    (same)

End result: Frames panel and Variables panel both refresh after every
step / resume cycle without the IDE having to explicitly re-suspend.

* Array inspection: ArrayReference.Length / GetValues + array type tag

ArrayList's `elementData` (and any other Object[] / int[] / etc.) was
showing as a single ref in the Variables view with no children — the
proxy stubbed both ArrayReference commands with NOT_IMPLEMENTED, so
IntelliJ couldn't expand the array.

Device side (cn1_debugger.m):
  - CMD_GET_ARRAY_LENGTH reads ((JAVA_ARRAY)obj)->length.
  - CMD_GET_ARRAY_VALUES walks ->data with element width chosen from
    primitiveSize + the element class's clsName (so byte/boolean
    arrays don't get conflated, neither do int/float and long/double).
    Element bytes go on the wire as packed big-endian — same layout
    JDWP expects for ArrayRegion primitive payloads.
  - CMD_GET_OBJECT_CLASS now reports the class struct's `isArray`
    flag as a trailing byte. Older proxies that only read 4 bytes
    still work.

Wire protocol additions: CMD_GET_ARRAY_LENGTH (0x0F),
CMD_GET_ARRAY_VALUES (0x10), EVT_ARRAY_LENGTH (0x8E),
EVT_ARRAY_VALUES (0x8F).

Proxy:
  - JdwpServer.handleArray wires JDWP ArrayReference.Length (cmd 1)
    and GetValues (cmd 2) to the device commands above. For object
    arrays the GetValues reply tags each element 'L' inline (JDWP's
    ArrayRegion encoding for non-primitive arrays); for primitives
    we forward the packed bytes verbatim.
  - DeviceConnection.onObjectClass now carries an isArray flag from
    the device. JdwpServer's ObjectReference.ReferenceType uses
    TYPE_TAG_ARRAY (3) for array instances so IntelliJ knows to
    issue ArrayReference commands rather than treating the object
    as a class instance and giving up.

Verified: dropping a BP in user code, expanding an ArrayList in the
Variables view now shows `elementData[0..size-1]` with values, not
just an opaque object reference.

* Docs: reflect array + invoke-method support, fix vale issues

Update the "What works today" list:
  - Add a bullet for array inspection (length + per-index values for
    Object[] and primitive arrays, including ArrayList.elementData
    drilldown).
  - Expand the method-invocation bullet to mention the
    Display.getInstance().getCurrent().getTitle() chain that motivated
    the implementation, plus the catch-all try block that keeps an
    uncaught throw from tearing down the session.

Vale: move punctuation inside the quoted phrase and rephrase the
first-person plural in the troubleshooting section.

* Docs: drive developer-guide LanguageTool count to zero on this branch

After merging master (which promoted LanguageTool to a hard quality
gate via #5007) the developer guide had 7 grammar-checker matches
attributable to this branch's chapter plus 1 pre-existing match
elsewhere. This drives the count back to 0.

Chapter-local fixes (docs/developer-guide/On-Device-Debugging.asciidoc):
  - Replace "source dir" with "source directory" so LanguageTool's
    morfologik English dictionary stops flagging the abbreviation.
  - Rephrase the "Watching expressions follow the same rule" sentence
    so QUESTION_MARK no longer fires (LanguageTool reads the "ing"
    opening as a rhetorical fragment expecting a question mark).
  - Rephrase the "What doesn't work:" lead so the wh-word doesn't
    trip QUESTION_MARK on the surrounding sentence.

Accept-list additions (docs/developer-guide/languagetool-accept.txt):
  - jdb (the JDK command-line debugger),
  - loopback (the networking term),
  - rethrow / rethrows (standard exception-handling vocabulary).
  These are well-known technical terms LanguageTool's default
  dictionary doesn't recognise but appear unaltered in any
  reasonable developer-facing prose.

Pre-existing match (docs/developer-guide/Maven-Getting-Started.adoc):
  Replace "(aka X. aka Y)" with "(also known as X or Y)" — the
  period between the two "aka"s made LanguageTool flag the second
  "aka" as a sentence-start lowercase letter. Master's docs job is
  PR-trigger-only so this never showed up there; it surfaced here
  because the merge brought in the gate.

* On-device debug: ship IDE configs in archetype + IDE-first dev guide

Wire on-device debugging into the cn1app-archetype so a fresh project
already has everything needed for the IDE-driven flow, not just the
command-line one:

- common/codenameone_settings.properties carries the four
  `ios.onDeviceDebug.*` build hints commented out, with a one-line
  pointer to the developer guide. Discoverable from the file every
  Codename One developer already edits.
- archetype-resources/.idea/runConfigurations/CN1_Debug_Proxy.xml is
  a Maven run config that invokes `codenameone:ios-on-device-debugging`
  from the project root — same proxy, same defaults, no path
  hardcoding.
- archetype-resources/.idea/runConfigurations/CN1_Attach_iOS.xml is a
  Remote JVM Debug config attaching to `localhost:8000`, scoped to
  the `${rootArtifactId}-common` module so IntelliJ's source
  resolution lines up with the user's code.
- .gitignore un-ignores the archetype's .idea/ subtree so these
  configs stay versioned (it still ignores everyone else's local
  .idea/).

Developer guide:
- Re-orders so the IntelliJ Quick start leads, the Maven /
  command-line + jdb flow becomes a smaller section after it. The
  text now explicitly says to **Run** (not Debug) the proxy config
  to avoid accidentally attaching IntelliJ's own debugger to the
  proxy process.
- Renames "iOS Simulator" to "native iOS simulator" wherever the
  Codename One simulator could otherwise be implied.
- Rewrites the "Working from a clone of this repository" bullet —
  promotes the Maven sources-jar approach (the path almost every
  reader will take), explains the cloned-CodenameOne-repo alternative
  by name, and links to the GitHub repository instead of implying
  the reader is already inside it.

Misc:
- `cn1_debugger.h` now wraps the `cn1_globals.h` include inside the
  `CN1_ON_DEVICE_DEBUG` ifdef so release builds don't pull in the
  ParparVM globals header at all.
- Single Microsoft.Auto vale warning fixed ("auto-resume" →
  "resume automatically").

The build-hints table already carried entries for
`ios.onDeviceDebug` and the three companion hints; updated one
parenthetical to say "native iOS simulator" for consistency.

* Archetype: actually check in the IDE run configs

The previous commit added the two run-config XMLs under
archetype-resources/.idea/runConfigurations/ but git silently dropped
them because of a second `.idea/` line further down in .gitignore that
re-applied the ignore after my un-ignore exception. Move the exception
below both ignore rules so the files actually land in the tree.
shai-almog added a commit that referenced this pull request May 23, 2026
`<URL-encoded-uri>` inside a backtick URL got interpreted by asciidoctor
as a partial HTML entity sequence (`&chl=<URL-encoded-uri>` produced an
unterminated `&` entity reference), failing the build under
`--failure-level WARN`. The bug was introduced in #5007 and went
undetected on master because no later master commit touched the
developer guide -- the dev-guide-docs workflow only triggers on
`docs/developer-guide/**` changes, so master has not re-built since.
This PR was the first to surface it.

Replace the literal `<...>` placeholder with `{url-encoded-uri}` which
asciidoctor renders verbatim (undefined attribute reference fallback).
Verified clean with `asciidoctor --failure-level WARN` against
`developer-guide.asciidoc`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant