Error loading native node module after upgrading from 0.6.0 to 1.0.2

[pdehne]

• Operating System: Windows
• Node Version: 12.6.3
• NPM Version: 7.6.0
• webpack Version: 5.24.2
• node-loader Version: 1.0.2

Expected Behavior

I am using a custom native node module in my Electron application. When using node-loader 0.6.0 it is loading fine. When using node-loader 1.0.2 with the very same node module (no recompilation) I get an error.

Actual Behavior

The error thrown is:

soniccore.node:5 Uncaught Error: node-loader:
Error: Invalid package C:\d\sw\trunk\SonicCare\node_modules\electron\dist\resources\electron.asar
at Object../build/Release/soniccore.node (soniccore.node:5)

Code

// webpack.config.js

target: "electron-renderer",

{
    test: /\.node$/,
    loader: 'node-loader',
    options: {
        name: '[path][name].[ext]'
    }
}

// The complete webpack.config.js:
// https://gist.github.com/pdehne/54a6a6f91435e332c70d5e00abeed20f

import soniccore from '../build/Release/soniccore.node';

[alexander-akait]

Can you create simple reproducible test repo?

[pdehne]

Sure. Here is a test repo: https://github.com/pdehne/node-loader-test

To reproduce please open a console in the project folder of the repo and run:

npm install
npm run configureaddon
npm run buildaddon
npm run watch

Then open a second console in the main project folder and run:
npm run start:watch

You will see the error message in the Electron console window:

Stop both npm processes using CTRL-c then change node-loader versions:

npm remove node-loader
npm install node-loader@0.6.0 --save-dev

Start watch / start:watch in their respective consoles again. The native node module is loaded successfully and you will see "Hello World from Node.js" in the Electron console:

[alexander-akait]

Weird, using:

npm remove node-loader
npm install node-loader@0.6.0 --save-dev

still have the problem:

Uncaught Error: Cannot open /path/to/node-loader-testazazaq/build/Release/nodeloadertest.node: Error: Module did not self-register: '/home/evilebottnawi/IdeaProjects/node-loader-testazazaq/build/Release/nodeloadertest.node'.

Maybe something buggy with cpp code?

[pdehne]

Did you run npm run configureaddon / npm run buildaddon? The repo targets Electron 10.1.1 and will only build the addon using the right target version if you use these commands.

[alexander-akait]

Did you run npm run configureaddon / npm run buildaddon?

Yes

Can you try other version of Electron or node?

[pdehne]

I upgraded the repo to Electron 11.3.0. I also streamlined the installation process to show first that it is working with 0.6.0. Please clone the repo into a new folder then run:

npm install
npm run watch

Open a second console and run

npm run start:watch

You should see "Hello World from Node.js" in the Electron console, just like in the screenshot above. If this works please first abort the two npm commands. Then, in one of the consoles run:

npm remove node-loader
npm install node-loader@latest

And again run npm run watch / npm run start:watch in the two consoles. You should see the error message from the screenshot above.

Can you reproduce?

[alexander-akait]

Thanks, I will look at this in near future

[alexander-akait]

Problem here:
https://github.com/webpack-contrib/node-loader/blob/master/src/index.js#L32:

• __dirname is /path/to/node-loader/node_modules/electron/dist/resources/electron.asar/renderer
• publicPath is http://localhost:8084/
• name is build/Release/nodeloadertest.node

We contact them so you just have invalid path. But here two weird things - __dirname and publicPath, maybe we can generate relative paths to module.

Just interesting why you need this loader, because you can replace this loader using:

try {
  process.dlopen(module, path.resolve(__dirname, './path/to/file.ext'));
} catch (error) {
   // logging
}

To be honestly we should deprecate it in favor simple code above

[vjpr]

__dirname seems to be set incorrectly. There are some workarounds here but I couldn't get them to work reliably.

[Bessonov]

I ran into a similar issue. The components are:

__dirname:               /home/dev/app/packages/main-backend/dist
__webpack_public_path__: file:///home/dev/app/packages/main-backend/dist/
name:                    50bcfcb41adcd2f723557101315cd073.node

And the error:

main-backend-1  | Error: node-loader:
main-backend-1  | Error: /home/dev/app/packages/main-backend/dist/file:/home/dev/app/packages/main-backend/dist/50bcfcb41adcd2f723557101315cd073.node: cannot open shared object file: No such file or directory
main-backend-1  |     at Object.../../.cache/pnpm/virtual-store-dir/@img+sharp-linux-x64@0.34.1/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64.node (file:///home/dev/app/packages/main-backend/dist/file:/home/dev/app/.cache/pnpm/virtual-store-dir/@img+sharp-linux-x64@0.34.1/node_modules/@img/sharp-linux-x64/lib/sharp-linux-x64.node:8:1)

It works better if I strip the result of join up to the colon with .replace(/^[^:]*:/, ""):

  return `
try {
  process.dlopen(module, require("path").join(__dirname, __webpack_public_path__, ${JSON.stringify(name)}).replace(/^[^:]*:/, "")${typeof options.flags !== "undefined" ? `, ${JSON.stringify(options.flags)}` : ""});
} catch (error) {
  throw new Error('node-loader:\\n' + error);
}
`;

@alexander-akait

To be honestly we should deprecate it in favor simple code above

Thanks! Could you please elaborate a bit more on that? I'm using Webpack to bundle the sources into a single server.mjs file, along with the *.node files, so I can store just a few MBs in the Docker image instead of ~700 MB of node_modules.

If I understand you correctly, you're suggesting that I should package the binary using asset/resource loader and then use something like:

import { dlopen } from 'node:process'
import library from 'librarypath.node' // returns path as string

const module = { exports: {} }
dlopen(module, library)

?

[alexander-akait]

@Bessonov Yeah, like:

const file = new URL("./file.node", import.meta.url);
const module = { exports: {} }

try {
  process.dlopen(module, file);
} catch (error) {
   // logging
}

console.log(module);

[Bessonov]

@alexander-akait Great, thanks! I can't try it out directly because it's used by a dependency, but it looks really good!

What do you think about my issue with the path?

[alexander-akait]

@Bessonov You should not have path issues here, because webpack copies *.node file and will use the valid path to the file

[Bessonov]

@alexander-akait thanks! I found the issue: despite setting target: 'node', I still needed to set an empty publicPath:

output: {
	publicPath: '',