Including node_modules during deno compile causes panic

[justinmchase]

> deno task build
Task build RUST_BACKTRACE=full deno compile --allow-import --allow-net --allow-env --allow-ffi --include=node_modules --output=./handler main.ts
Compile file:///home/justin/code/scantact/func/main.ts to ./handler

============================================================
Deno has panicked. This is a bug in Deno. Please report this
at https://github.com/denoland/deno/issues/new.
If you can reliably reproduce this panic, include the
reproduction steps and re-run with the RUST_BACKTRACE=1 env
var set and include the backtrace in your report.

Platform: linux x86_64
Version: 2.2.8
Args: ["/home/justin/.deno/bin/deno", "compile", "--allow-import", "--allow-net", "--allow-env", "--allow-ffi", "--include=node_modules", "--output=./handler", "main.ts"]

View stack trace at:
https://panic.deno.com/v2.2.8/x86_64-unknown-linux-gnu/m-g8tEu8_06Fgzqm7Fgyqm7Fgn4m7Fgh7rwEggnswEgw289Eqi-89E23n99Eqvn99EyzlorEwihwrEujt4tEm0x4kEoml8tEmvi8hEgmi8uEg5_IggwltCA

thread 'main' panicked at cli/lib/standalone/virtual_fs.rs:612:14:
internal error: entered unreachable code
stack backtrace:
   0:     0x565531624439 - <unknown>
   1:     0x5655301001b3 - <unknown>
   2:     0x565531623492 - <unknown>
   3:     0x565531624293 - <unknown>
   4:     0x565531623a36 - <unknown>
   5:     0x56552ffbf52e - <unknown>
   6:     0x5655316231ac - <unknown>
   7:     0x565531668595 - <unknown>
   8:     0x565531668529 - <unknown>
   9:     0x56553166a08c - <unknown>
  10:     0x5655300fe62f - <unknown>
  11:     0x5655300ffe3b - <unknown>
  12:     0x5655307c1f5f - <unknown>
  13:     0x5655307c2ddd - <unknown>
  14:     0x5655307c4063 - <unknown>
  15:     0x5655307c3f6f - <unknown>
  16:     0x56552fe70c16 - <unknown>
  17:     0x56552fe90d9b - <unknown>
  18:     0x56552ffb0beb - <unknown>
  19:     0x56552fb313c4 - <unknown>
  20:     0x56552ffc12fc - <unknown>
  21:     0x56552f9bf4f9 - <unknown>
  22:     0x56553003f8e5 - <unknown>
  23:     0x7fab83d0e083 - __libc_start_main
  24:     0x56552df65029 - <unknown>
  25:                0x0 - <unknown>

Reproduction

I think the minimal for this would be:

1. deno add npm:canvas
2. Create a main.ts with import nodeCanvas from "canvas"; in it
3. deno compile --allow-import --allow-net --allow-env --allow-ffi --include=node_modules --output=./handler main.ts

[littledivy]

Side note: The panic.deno.com link doesn't work because of a bug in the release process. Fix in #28774

Stack trace

Deno v2.2.8+1086a75 on aarch64-apple-darwin.

stack backtrace:

• deno::setup_panic_hook::{{closure}}
• std::panicking::rust_panic_with_hook
• std::panicking::begin_panic_handler::{{closure}}
• std::panicking::begin_panic_handler::{{closure}}
• std::os::fd::owned::BorrowedFd::borrow_raw
• <std::os::fd::owned::OwnedFd as std::os::fd::owned::AsFd>::as_fd
• <std::sys::pal::unix::fd::FileDesc as std::os::fd::owned::AsFd>::as_fd
• core::fmt::Formatter::pad_integral
• core::panicking::assert_failed
• deno_lib::standalone::virtual_fs::VfsBuilder::add_dir_raw
• deno_lib::standalone::virtual_fs::VfsBuilder::add_file_with_data_raw_options
• deno_lib::standalone::virtual_fs::VfsBuilder::add_file_with_data
• deno_lib::standalone::virtual_fs::VfsBuilder::add_file_at_path
• deno::standalone::binary::DenoCompileBinaryWriter::write_standalone_binary::{{closure}}
• {async_fn#0}
• deno::tools::compile::compile::{{closure}}

_{View trace: here}

[dsherret]

I think this is because it's including the node_modules directory twice because it's already automatically included. Probably dropping --include=node_modules will fix it.

[justinmchase]

@dsherret Ok got it. I added that because if you try to run it with the import of canvas it gives an error saying it can't find the canvas.node file that is in the node modules folder.

Did your fix also fix that by chance? Or would that be a different issue?

[dsherret]

@justinmchase the install needs to be run with --allow-scripts to run the lifecycle scripts as the warning suggests here:

> deno add npm:canvas
Add npm:[email protected]
Warning The following packages contained npm lifecycle scripts (preinstall/install/postinstall) that were not executed:
┠─ npm:[email protected]
┃
┠─ This may cause the packages to not work correctly.
┖─ To run lifecycle scripts, use the `--allow-scripts` flag with `deno install`:
   deno install --allow-scripts=npm:[email protected]
V:\scratch
> deno run -A main.ts
error: Uncaught (in promise) Error: Cannot find module '../build/Release/canvas.node'
Require stack:
- ...

    info: Trying to execute an npm package using Node-API addons,
          these packages require local `node_modules` directory to be present.
    hint: Add `"nodeModulesDir": "auto" option to `deno.json`, and then run
          `deno install --allow-scripts=npm:<package> --entrypoint <script>` to setup `node_modules` directory.
> deno install --allow-scripts=npm:[email protected]
> deno run -A main.ts
{
  Canvas: [Function: Canvas] {
...
}
> deno compile -A main.ts  
...
> ./scratch.exe
scratch.exe\node_modules\.deno\[email protected]\node_modules\canvas\build\Release\canvas.node".
error: Uncaught (in promise) TypeError: LoadLibraryExW failed
    at Object.Module._extensions..node (node:module:805:20)
    at Module.load (node:module:662:32)
    at Module._load (node:module:534:12)
    at Module.require (node:module:681:19)
    at require (node:module:816:16)
    at Object.<anonymous> (file:///C:/Users/david/AppData/Local/Temp/deno-compile-scratch.exe/node_modules/.deno/[email protected]/node_modules/canvas/lib/bindings.js:3:18)
    at Object.<anonymous> (file:///C:/Users/david/AppData/Local/Temp/deno-compile-scratch.exe/node_modules/.deno/[email protected]/node_modules/canvas/lib/bindings.js:45:4)
    at Module._compile (node:module:745:34)
    at loadMaybeCjs (node:module:770:10)
    at Object.Module._extensions..js (node:module:755:12)

Note that node native modules doesn't work in deno compile atm, so that's why the deno compile output errors here. Follow #23266 for updates

[justinmchase]

Right, I do include --allow-scripts and am able to run it locally just fine it's only when compiled that it appears to error out reporting that the .node file is missing.

I guess I'm just trying to report that one of the issues with it not working at the moment appears to be that the file is not copied when compiled.

Thanks for looking into this!

[dsherret]

I guess I'm just trying to report that one of the issues with it not working at the moment appears to be that the file is not copied when compiled.

I see it included as shown above. Can you try in canary? deno upgrade canary

I excluded the deno compile output above actually, but it shows me this (so including the entire node_modules directory):

> deno compile main.ts  
Check file:///V:/scratch/main.ts
Compile file:///V:/scratch/main.ts to scratch.exe

Embedded Files

scratch.exe
├── main.ts (217B)
├── node_modules/* (31.68MB - 31.67MB unique)
└── package.json (269B)

Files: 31.73MB
Metadata: 1.6KB
Remote modules: 12B