deno compile binaries should not include @types/node in the npm virtual filesystem

[andreubotella]

Deno 1.34 added support for npm imports in deno compiled binaries by adding a virtual filesystem containing the contents of the node_modules folder that gets generated when walking through the modules at compile time. This node_modules folder should only include npm modules directly imported by Deno modules, as well as their recursive dependencies. However, this virtual filesystem also includes the @types/node package, even though the compiled binary does not transpile Typescript, let alone typecheck it.

This presumably happens because the virtual filesystem is built from the node_modules folder built when the module graph is walked through at compile time, which does do typechecking. However, including the node_modules folder directly into the vfs just results in unnecessary extra weight in the binary that could be saved with not too much trouble by pruning packages that aren't recursive dependencies of those imported by Deno code.

[dsherret]

Yeah, we need to get the npm packages from the module graph, then only include those and their descendant deps in the final output. Right now we include everything found in the npm snapshot, which could include extra dependencies found in the lockfile.

[Antman261]

I don't have any npm modules in my code, yet I'm still seeing @types/node in my compiled binary? According to the console output from deno compile, this is adding 2.1MB to my binary, accounting for 92% of its size.

I'm using Deno v2.1.7

[dsherret]

It will include whatever is in the lockfile at the moment to make more stuff work, which is not ideal. Follow #27305 for one potential mitigation.

[Antman261]

Is @types/node something Deno adds to the lock file? It's not actually in my deno.lock file (I deleted it) but it is still appearing in the output from the compile command 🤔 It also wasn't a dependency of anything -- there was an npm section of the lock file but nothing needing either of the packages listed there

[dsherret]

@Antman261 maybe if you are using a node: import somewhere or one of your dependencies uses it?

[dsherret]

Note that it's somewhat possible to exlude this when using a node_modules directory when using the --exclude flag and providing something like --exclude node_modules/.deno/@[email protected] (dependening on the name of the package you want to exclude), but it's not great because it's verbose.

[kingbri1]

@dsherret I was forwarded to this issue and tried that solution, the compiled binary is still the same size and the module graph shows @types/node post-compilation. Am I doing something wrong here?

My command: deno compile -A --exclude node_modules/.deno/@[email protected] main.ts

The only node API I use with my deno project is node:process.

[dsherret]

Huh, ok I had to run this absolutely terrible command (basically ensure the symlinks aren't included as well, which seems to cause it to include where they point to):

> deno compile --exclude node_modules/.deno/@[email protected]/ --exclude node_modules/.deno/[email protected] --exclude node_modules/@types --exclude node_modules/.deno/node_modules/undici-types main.ts
Compile file:///V:/scratch/main.ts to scratch.exe

Embedded Files

scratch.exe
├── main.ts (313B)
└─┬ node_modules (43.81KB - 43.78KB unique)
  ├── .bin/* (0B)
  ├─┬ .deno (43.81KB - 43.78KB unique)
  │ ├── .deno.lock (12B)
  │ ├── .deno.lock.poll (13B - 1B unique)
  │ ├── .setup-cache.bin (429B)
  │ ├── [email protected]/* (43.36KB - 43.35KB unique)
  │ └── node_modules (0B)
  └── chalk --> node_modules/.deno/[email protected]/node_modules/chalk

Files: 47.07KB
Metadata: 1.42KB
Remote modules: 61B

It's not pretty, but it works for the time being.

You might try running with --no-check (related bug here: #28254)

[kingbri1]

Ok, I figured out the problem I was having. I apparently needed to add the "node_modules_dir": "auto" option in my deno.json (which is what this issue was about in the first place XD).

After doing that, the command worked fine. However, I'd ideally not like to make a temp node_modules dir and have this work with just an invocation of deno compile. Hopefully the PR linked to this issue can solve the problem.