local_repository installs node_modules in location not reachable by tsconfig.json

[mhevery]

🐞 bug report

Affected Rule

build_bazel_rules_nodejs

Is this a regression?

Yes, this used to work in 3.5.1

Description

When using nested repositories building from child repository works, but building from parent repository does not because the location of the node_modules is in place which is not visible from tsconfig.json. See repro steps for details.

🔬 Minimal Reproduction

https://github.com/mhevery/repro-bazel-nested-project

Bazel rules_nodejs issue reproduction

Steps to reproduce:

cd child
bazel build packages/core

Above succeeds. This proves that the ./child local_repository is correctly configured.

Next:

cd ..
bazel build @child//packages/core

The above command fails with:

INFO: Analyzed target @child//packages/core:core (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_misko/7cb4da9d45aa96b44a2c9ccbd78ae6c8/external/child/packages/core/BUILD:5:11: Compiling TypeScript project @child//packages/core:core [tsc -p ../child/tsconfig.json] failed: (Exit 2): tsc.sh failed: error executing command bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/npm_child/typescript/bin/tsc.sh --project external/child/tsconfig.json --outDir bazel-out/darwin_arm64-fastbuild/bin/external/child/packages/core ... (remaining 5 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox
external/child/packages/core/main.ts(1,19): error TS2307: Cannot find module 'is-odd' or its corresponding type declarations.
Target @child//packages/core:core failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.287s, Critical Path: 0.56s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully```

The core issue is this:

external/child/packages/core/main.ts(1,19): error TS2307: Cannot find module 'is-odd' or its corresponding type declarations.

It seems that the typescript compiler can't find node_modules to resolve is-odd.

Running with --sandbox_debug reveals this layout:

execroot/parent
  - bazel-out
  - external
    - child
      - tsconfig.json
      - packages
        - core
          - main.ts
  - node_modules <= @npm
    - typescript
  - packages
    - core
      - node_modules <= @npm_child_core
        - is-odd

The core issue is that from typescript perspective the node_module which contains is-odd is buried in such a way that it can't be found. I am not aware of any way one could update tsconfig.json so that it can find the buried node_module with is-odd

🔥 Exception or Error

INFO: Analyzed target @child//packages/core:core (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
ERROR: /private/var/tmp/_bazel_misko/7cb4da9d45aa96b44a2c9ccbd78ae6c8/external/child/packages/core/BUILD:5:11: Compiling TypeScript project @child//packages/core:core [tsc -p ../child/tsconfig.json] failed: (Exit 2): tsc.sh failed: error executing command bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/external/npm_child/typescript/bin/tsc.sh --project external/child/tsconfig.json --outDir bazel-out/darwin_arm64-fastbuild/bin/external/child/packages/core ... (remaining 5 arguments skipped)

Use --sandbox_debug to see verbose messages from the sandbox
external/child/packages/core/main.ts(1,19): error TS2307: Cannot find module 'is-odd' or its corresponding type declarations.
Target @child//packages/core:core failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 1.287s, Critical Path: 0.56s
INFO: 2 processes: 2 internal.
FAILED: Build did NOT complete successfully```

🌍 Your Environment

Operating System:

  
MacBook Pro (13-inch, M1, 2020) v12.1
  

Output of bazel version:

  
Bazelisk version: development
Build label: 5.0.0
Build target: bazel-out/darwin_arm64-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Wed Jan 19 14:16:16 2022 (1642601776)
Build timestamp: 1642601776
Build timestamp as int: 1642601776
  

Rules_nodejs version:

(Please check that you have matching versions between WORKSPACE file and @bazel/* npm packages.)

  
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "c077680a307eb88f3e62b0b662c2e9c6315319385bc8c637a861ffdbed8ca247",
    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/5.1.0/rules_nodejs-5.1.0.tar.gz"],
)
  

Anything else relevant?

[alexeagle]

Yeah I think this is a missing linker feature. We never took into account that the target being built is not in the current repository, but is in some external one. I think we need to place a node_modules tree inside the execroot/parent/external/child folder. It also gets more complex depending on the execroot layout.
Will probably need @gregmagolan to look at it, I'm not that familiar in the linker code for the last year or so.

[mhevery]

I assume that linker is runtime issue. This fails during compilation so before the linker.

…

On Sat, Feb 12, 2022 at 12:22 PM Alex Eagle ***@***.***> wrote: Yeah I think this is a missing linker feature. We never took into account that the target being built is not in the current repository, but is in some external one. I think we need to place a node_modules tree inside the execroot/parent/external/child folder. It also gets more complex depending on the execroot layout. Will probably need @gregmagolan <https://github.com/gregmagolan> to look at it, I'm not that familiar in the linker code for the last year or so. — Reply to this email directly, view it on GitHub <#3323 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAA3KTZT6DC7T57L3X7BSRDU226QZANCNFSM5OGARNVA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mhevery]

I have downgraded to v4.6.1 and it works correctly: https://github.com/mhevery/repro-bazel-nested-project/tree/bazel_4.6.1
I then upgrade to v5.0.0 and it no longer works: https://github.com/mhevery/repro-bazel-nested-project/tree/bazel_5.0.0

So the regression must have been introduced someplace between those two releases.