Xcode detection issue

[tiennou]

$ node-gyp -v
v3.8.0
$ sw_vers
ProductName:		macOS
ProductVersion:		13.7.4
BuildVersion:		22H420
$ node -v
v23.6.0
$ xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" in System Settings to install updates
$ cc -v
Apple clang version 14.0.3 (clang-1403.0.22.14.1)
Target: arm64-apple-darwin22.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

I've been trying to install a project using Sequelize, which needs to build its own sqlite library. Error log included below.

Since that's giving me an error, I've studied things a little (I have never been near node-gyp, so I do not know what I'm doing there). I don't see anything in Sequelize's bindings.gyp file that makes me think it has a hard dependency on Xcode, especially since I already have it installed through Homebrew, so I know the CommandLineTools are sufficient.

Looking deeper, I had a look at what was causing the errors in node-gyp. As it turns out, I'm failing all the checks for the compile tools; no Xcode, which is fair since I don't have that installed, but more concerning to me is no CommandLineTools. So I tried looking closer at which part is failing, and there's a few things I notice in CLTVersion:

• /usr/sbin/pkgutil --pkg-info checks for a couple pkgId, none of which I have. I've looked over the complete list inpkgutils --pkgs doesn't give me anything about CLTs being there so I don't know where the receipt for that went but I'm clearly missing it.
• /usr/sbin/softwareupdate --history shows me nothing about CLTs as well.

So my guess is that the CLT detection code is somehow broken here (I see there's a comment saying it's a reimplementation of Homebrew). Yet Homebrew sees no problem with my install, given it can compile and install sqlite from source.

yarn install output

yarn install v1.22.22
[1/4] 🔍  Resolving packages...
[2/4] 🚚  Fetching packages...
[3/4] 🔗  Linking dependencies...
[4/4] 🔨  Building fresh packages...
error /Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3: Command failed.
Exit code: 1
Command: node-pre-gyp install --fallback-to-build
Arguments: 
Directory: /Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3
Output:
node-pre-gyp info it worked if it ends with ok
node-pre-gyp info using node-pre-gyp@0.11.0
node-pre-gyp info using node@23.6.0 | darwin | arm64
node-pre-gyp WARN Using request for node-pre-gyp https download 
(node:19681) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
node-pre-gyp info check checked for "/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/lib/binding/napi-v6-darwin-arm64/node_sqlite3.node" (not found)
node-pre-gyp http GET https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-darwin-arm64.tar.gz
node-pre-gyp http 403 https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-darwin-arm64.tar.gz
node-pre-gyp WARN Tried to download(403): https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-darwin-arm64.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for sqlite3@5.0.1 and node@23.6.0 (node-v131 ABI, unknown) (falling back to source compile with node-gyp) 
node-pre-gyp http 403 status code downloading tarball https://mapbox-node-binary.s3.amazonaws.com/sqlite3/v5.0.1/napi-v6-darwin-arm64.tar.gz 
gyp info it worked if it ends with ok
gyp info using node-gyp@11.0.0
gyp info using node@23.6.0 | darwin | arm64
gyp info ok 
gyp info it worked if it ends with ok
gyp info using node-gyp@11.0.0
gyp info using node@23.6.0 | darwin | arm64
gyp info find Python using Python version 3.13.1 found at "/opt/homebrew/opt/python@3.13/bin/python3.13"

gyp info spawn /opt/homebrew/opt/python@3.13/bin/python3.13
gyp info spawn args [
gyp info spawn args '/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args 'binding.gyp',
gyp info spawn args '-f',
gyp info spawn args 'make',
gyp info spawn args '-I',
gyp info spawn args '/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/build/config.gypi',
gyp info spawn args '-I',
gyp info spawn args '/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/addon.gypi',
gyp info spawn args '-I',
gyp info spawn args '/Users/tiennou/Library/Caches/node-gyp/23.6.0/include/node/common.gypi',
gyp info spawn args '-Dlibrary=shared_library',
gyp info spawn args '-Dvisibility=default',
gyp info spawn args '-Dnode_root_dir=/Users/tiennou/Library/Caches/node-gyp/23.6.0',
gyp info spawn args '-Dnode_gyp_dir=/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp',
gyp info spawn args '-Dnode_lib_file=/Users/tiennou/Library/Caches/node-gyp/23.6.0/<(target_arch)/node.lib',
gyp info spawn args '-Dmodule_root_dir=/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3',
gyp info spawn args '-Dnode_engine=v8',
gyp info spawn args '--depth=.',
gyp info spawn args '--no-parallel',
gyp info spawn args '--generator-output',
gyp info spawn args 'build',
gyp info spawn args '-Goutput_dir=.'
gyp info spawn args ]
No receipt for 'com.apple.pkg.CLTools_Executables' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLILeo' found at '/'.

No receipt for 'com.apple.pkg.DeveloperToolsCLI' found at '/'.

Traceback (most recent call last):
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py", line 1501, in XcodeVersion
    version_list = GetStdoutQuiet(["xcodebuild", "-version"]).splitlines()
                   ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py", line 1562, in GetStdoutQuiet
    raise GypError("Error %d running %s" % (job.returncode, cmdlist[0]))
gyp.common.GypError: Error 1 running xcodebuild

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/gyp_main.py", line 45, in <module>
    sys.exit(gyp.script_main())
             ~~~~~~~~~~~~~~~^^
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 688, in script_main
    return main(sys.argv[1:])
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 680, in main
    return gyp_main(args)
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/__init__.py", line 665, in gyp_main
    generator.GenerateOutput(flat_list, targets, data, params)
    ~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/generator/make.py", line 2682, in GenerateOutput
    writer.Write(
    ~~~~~~~~~~~~^
        qualified_target,
        ^^^^^^^^^^^^^^^^^
    ...<4 lines>...
        part_of_all=qualified_target in needed_targets,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/generator/make.py", line 893, in Write
    self.WriteActions(
    ~~~~~~~~~~~~~~~~~^
        spec["actions"],
        ^^^^^^^^^^^^^^^^
    ...<3 lines>...
        part_of_all,
        ^^^^^^^^^^^^
    )
    ^
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/generator/make.py", line 1027, in WriteActions
    env = self.GetSortedXcodeEnv()
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/generator/make.py", line 2279, in GetSortedXcodeEnv
    return gyp.xcode_emulation.GetSortedXcodeEnv(
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self.xcode_settings,
        ^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
        additional_settings,
        ^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py", line 1887, in GetSortedXcodeEnv
    env = _GetXcodeEnv(
        xcode_settings, built_products_dir, srcroot, configuration, additional_settings
    )
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py", line 1742, in _GetXcodeEnv
    "XCODE_VERSION_ACTUAL": XcodeVersion()[0],
                            ~~~~~~~~~~~~^^
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py", line 1512, in XcodeVersion
    version = CLTVersion()  # macOS Catalina returns 11.0.0.0.1.1567737322
  File "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/gyp/pylib/gyp/xcode_emulation.py", line 1550, in CLTVersion
    return re.search(regex, output).groupdict()["version"]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'groupdict'
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack at ChildProcess.<anonymous> (/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:317:18)
gyp ERR! stack at ChildProcess.emit (node:events:507:28)
gyp ERR! stack at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.6.0
gyp ERR! command "/opt/homebrew/Cellar/node/23.6.0/bin/node" "/opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/lib/binding/napi-v6-darwin-arm64/node_sqlite3.node" "--module_name=node_sqlite3" "--module_path=/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/lib/binding/napi-v6-darwin-arm64" "--napi_version=10" "--node_abi_napi=napi" "--napi_build_version=6" "--node_napi_label=napi-v6"
gyp ERR! cwd /Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3
gyp ERR! node -v v23.6.0
gyp ERR! node-gyp -v v11.0.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/opt/homebrew/Cellar/node/23.6.0/bin/node /opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/lib/binding/napi-v6-darwin-arm64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/lib/binding/napi-v6-darwin-arm64 --napi_version=10 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (node:events:507:28)
node-pre-gyp ERR! stack     at maybeClose (node:internal/child_process:1101:16)
node-pre-gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:305:5)
node-pre-gyp ERR! System Darwin 22.6.0
node-pre-gyp ERR! command "/opt/homebrew/Cellar/node/23.6.0/bin/node" "/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3
node-pre-gyp ERR! node -v v23.6.0
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! not ok 
Failed to execute '/opt/homebrew/Cellar/node/23.6.0/bin/node /opt/homebrew/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/lib/binding/napi-v6-darwin-arm64/node_sqlite3.node --module_name=node_sqlite3 --module_path=/Users/tiennou/Projects/screeps/mods/screepsmod-history/node_modules/sqlite3/lib/binding/napi-v6-darwin-arm64 --napi_version=10 --node_abi_napi=napi --napi_build_version=6 --node_napi_label=napi-v6' (1)
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

[cclauss]

Please run:
% softwareupdate --help
% softwareupdate --list

This package?? https://www.npmjs.com/package/output

node-pre-gyp info using node-pre-gyp@0.11.0

Perhaps upgrade https://github.com/mapbox/node-pre-gyp/releases

[tiennou]

There you go:

$ softwareupdate --help
usage: softwareupdate <cmd> [<args> ...]

** Manage Updates:
	-l | --list		List all appropriate update labels (options:  --no-scan, --product-types)
	-d | --download		Download Only
	-i | --install		Install

        [ ... ]

	--verbose		Enable verbose output
	--help			Print this help

$ softwareupdate --list
Software Update Tool

Finding available software
Software Update found the following new or updated software:
* Label: macOS Sequoia 15.3.1-24D70
	Title: macOS Sequoia 15.3.1, Version: 15.3.1, Size: 9549726KiB, Recommended: YES, Action: restart, 

I kinda managed to workaround it by just returning a random string from CLTVersion (IIRC return "20.0.0") and after fighting with my asdf install a bit did manage to get it to install properly.

The thing I didn't really say is that Screeps — the thing I'm installing that screepsmod-history mod for — is stuck on Node 10, so it's not like I can update a lot of things, or switch to something that doesn't compile against that. I just wanted to point out that somehow the version checks in node-gyp don't work on my machine.
EDIT Actually, not completely true; I am installing on my local, modern Node, only because I want to be able to use VSCode's code navigation and typechecking abilities properly, hence I need the package to install its deps.

Something else I've tried is moving aside the /Library/Developer/CommandLineTools and running xcode-select --install, but that's broken; I get the CoreService app running but it froze when I click Install.

[cclauss]

https://github.com/nodejs/node-gyp/blob/main/macOS_Catalina_acid_test.sh is available on this repo but is probably of limited utility.

This file used to exist in the repo to cover these issues but Apple got its act together in later versions of macOS so it was removed. https://github.com/nodejs/node-gyp/blob/05ad2a11e09cae457e3515c3aba22b2c77e18a05/macOS_Catalina.md

Providing support for Node.js v10 is not really in scope.

[cclauss]

• yarn.lock: node-gyp@~3.6.0 will break builds on modern version of Node.js ScreepsMods/screepsmod-history#32