Java 代码不兼容

[yswwpp]

get_parser('java') 在 tree-sitter 0.25.2 + tree-sitter-language-pack 1.6.2 下死锁。

[stephantul]

So we're all on the same page:

get_parser('java') 在 tree-sitter 0.25.2 + tree-sitter-language-pack 1.6.2 下死锁。

Translates to:

The get_parser('java') function deadlocked on tree-sitter 0.25.2 + tree-sitter-language-pack 1.6.2.

Thanks for reporting this issue! Can you give more information? We already catch download errors thrown by tree sitter language pack, but I think you might just be hitting an issue with an internet connection not timing out. Does it ever time out? Is your use of the word deadlock literal? Or is execution simply blocked because the download never finishes?

[yswwpp]

2026年5月24日 15:21，Stephan Tulkens ***@***.***> 写道： stephantul left a comment (MinishLab/semble#151) <#151 (comment)> So we're all on the same page: get_parser('java') 在 tree-sitter 0.25.2 + tree-sitter-language-pack 1.6.2 下死锁。 Translates to: "The get_parser('java') function deadlocked on tree-sitter 0.25.2 + tree-sitter-language-pack 1.6.2." Thanks for reporting this issue! Can you give more information? We already catch downloaderrors, but I think you might just be hitting a time-out. Does it ever time out? Is your use of the word deadlock literal? Or is execution simply blocked because the download never finishes? — Reply to this email directly, view it on GitHub <#151 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AC4MC4BXJHIRAQNVLRUZHIL44KPJLAVCNFSM6AAAAACZKO2EPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DKMRXG4YDINZWGU>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you authored the thread.

Hi Stephan and everyone, Thanks for the detailed follow-up. Here are my answers to your questions: What happened When calling get_parser('java') or get_language('java') from tree-sitter-language-pack on macOS ARM64, the process hangs forever. No error, no exception, no timeout — it just stops. The only way out is kill -9. This breaks any tool that relies on tree-sitter for parsing Java code, including Semble <https://github.com/MinishLab/semble>, which uses it for code search indexing. Environment OS: macOS 15 (Darwin 24.6.0), Apple Silicon (ARM64) Python: 3.13 tree-sitter: 0.25.2 tree-sitter-language-pack: 1.6.2 Install method: pip install / uv tool install Reproduction Minimal code — no Semble, no project, no files needed: from tree_sitter_language_pack import get_parser ​ # Hangs forever on macOS ARM64 parser = get_parser("java") ​ # Works fine parser = get_parser("python") # returns instantly You can also reproduce via CLI: pip install tree-sitter==0.25.2 tree-sitter-language-pack==1.6.2 python -c "from tree_sitter_language_pack import get_parser; get_parser('java')" # hangs indefinitely It only happens on macOS ARM64 I tested the exact same package versions on Linux (both ARM64 and x86_64) using Docker containers. Java parser loads fine on both. Platform get_parser("java") Result macOS ARM64 (native) never returns HANG Linux ARM64 (Docker) 1.34s OK Linux x86_64 (Docker) 1.23s OK The Docker test was run with python:3.13-slim and the same tree-sitter==0.25.2 + tree-sitter-language-pack==1.6.2. Full test script available on request. It's not about project size or file count The hang happens during language parser initialization, not during file processing. Even a single .java file triggers it, because Semble calls get_parser('java') the first time it encounters any Java file and caches the result with @cache. I confirmed this step by step: Step Time Status load_model() (embedding model) 0.2s OK walk_files() (file traversal) 0.0s OK chunk_source() → get_parser('java') never returns HANG get_parser('python') 0.0s OK The deadlock is at the C/Rust level Python signals can't interrupt it. I tried signal.SIGALRM with a 8-second timeout — the handler never fires. The process can only be killed with SIGKILL. This points to a deadlock inside the Rust native extension (_native.abi3.so), not a Python-level issue. The cached dylib file is not corrupt $ ls -la ~/.cache/tree-sitter/lib/java.dylib -rwxr-xr-x 447288 May 8 10:46 java.dylib ​ $ file ~/.cache/tree-sitter/lib/java.dylib Mach-O 64-bit dynamically linked shared library arm64 ​ # Loading it directly via ctypes works fine $ python -c "import ctypes; ctypes.CDLL('~/.cache/tree-sitter/lib/java.dylib')" # returns in 0.1s, no problem I also tried deleting the cache and letting it re-download — same hang. Other language pack versions don't help tree-sitter tree-sitter-language-pack macOS ARM64 result 0.25.2 1.6.2 hang 0.24.0 1.6.2 hang 0.25.2 1.8.1 hang Workaround: use standalone tree-sitter-java The standalone tree-sitter-java package loads the Java parser instantly on macOS ARM64: import tree_sitter_java as tsjava from tree_sitter import Language, Parser ​ lang = Language(tsjava.language()) parser = Parser(lang) # returns in 0.0s This works as a drop-in replacement. In Semble's case, I patched semble/chunking/core.py to route Java through the standalone package: # In _cached_get_parser(): if language == "java": return Parser(_Language(_tsjava.language())) return get_parser(language) After this patch, Semble indexes the full project (1292 Java files, 10056 chunks) in 4.9 seconds on macOS ARM64, search returns instantly. Before the patch, it hung forever on even a single Java file. Impact on Semble This breaks Semble completely for any Java project on macOS ARM64: MCP mode: search requests time out with MCP error -32001 CLI mode: semble search "query" ./java-project hangs indefinitely All Java projects affected, regardless of size Related Semble issues: #151 <#151> — Java code incompatibility #133 <#133> — MCP server timeout on Java projects #135 <#135> — Tree-sitter chunker fails, no per-file isolation What I think is happening tree-sitter-language-pack ships a Rust native extension (_native.abi3.so on Linux, equivalent on macOS) that handles language loading. When get_language('java') is called on macOS ARM64, something inside this extension deadlocks — likely a mutex or initialization routine that doesn't play well with macOS's threading model or dynamic linking behavior. The fact that: It works on Linux (both architectures) The standalone tree-sitter-java package works on macOS The cached .dylib loads fine via ctypes Python signals can't interrupt it ...strongly suggests the bug is in the Rust extension's macOS-specific code path, not in the grammar or the compiled .dylib itself. Suggested fixes Short term: In Semble, add a per-language timeout or try/except around _cached_get_parser(). If loading a parser hangs, fall back to line-based chunking (already exists as chunk_lines()). Alternatively, use standalone tree-sitter-java as a fallback for Java specifically. Proper fix: The deadlock needs to be fixed in tree-sitter-language-pack's Rust extension for macOS ARM64. I don't have the Rust expertise to debug further, but I can provide any testing needed on my machine.

[stephantul]

Hello!

Thanks for the extensive benchmarks and experiments.

I think this is an issue in tree-sitter-language-pack, not in Semble. I can't reproduce it, nor have I experienced it. The related issues you cite are actually not related to these issues. Can you open an issue in the tree-sitter-language-pack? We'll see how it resolves and if we can fix it from our side.