Plans for npm 9

[MylesBorins]

Hey all. The npm team is hard at work on getting npm 9 together with the hope that we could have the version of npm included in Node.js 18 prior to LTS. A big part of wanting to do this is to ensure we are in a good place for maintenance of the CLI with 18 LTS being supported until 2025.

We are intending to start cutting release candidates for npm 9 in the next week or two. Is there a certain amount of time prior to the October 25th LTS date that folks would want to see us release a GA of npm 9 to feel comfortable including it in node.js 18?

[mhdawson]

Are there no breaking changes in npm 9 even though its a major? I think by default our policy is no breaking change once we've cut the major.

[MylesBorins]

@mhdawson the breaking changes can be seen in the link I shared above

npm/statusboard#443

There are a number of them, but none that seem to be extremely disruptive to folks for the mainstay use case of "publish" and "install".

We have prior art of updating npm Semver-Majors during a release line. Node.js 16 was updated from npm 7 -> npm 8 while it was current in v16.11.0 in early october of 2021 (a similar timeline to now). It included a breaking change of dropping support to the require('npm') interface, changing support matrix, and updating dependencies to drop support for node 10.

From a stability stand point I think it is the right thing to do for the ecosystem to ensure that the latest Node.js LTS is using a version of npm that will be getting regular bug fixes and features.

[MylesBorins]

as a caveat perhaps we need to have a broader conversation about versioning of npm in node.js, and our release schedules to try and come up with an agreement for future LTS + Majors to avoid needing to revisit this conversation each year (or major) 😓

[mhdawson]

@nodejs/release I think an opinion from the release team would be good and adding @nodejs/tsc in case as well for awareness.

[mcollina]

I find it unfortunate that npm decided to ship a major release the same month Node hit lts.

I think it's time we have this conversation of aligning the releases timelines. I think the best thing is for npm to release a major in March or April, so that there is plenty of time for all bugs to be fixed before then.

My 2 cents: there is a long list of breaking changes that still need to be written on user configs etc likely to cause havoc without some proper testing. I don't want to test in LTS.

Given all the above, I’m -1 in shipping Npm v9 in a Node LTS release.

[mhdawson]

@mcollina I like the suggestion of aligning LTS releases between Node and npm. If there was a major npm release in March or April that would be a great way to get a recent major into the Node.js version that will be the next LTS along with enough backing time to minimize impact/surprises for the ecosystem.

@MylesBorins I know that is probably a separate discussion from the current ask, but do you think that would be possibe in the future?

[MylesBorins]

I'm 100% on board with moving forward with improving coordination for future releases.

For the sake of Node.js 18 and thinking about long term stability and feature compatibility I think it is critical we land npm 9. We are putting together an explanation of the changes and I'd be willing to push back on the npm team to delay some of these changes to a future major release if it would allow us to be aligned on landing npm 9 in node.js 18

[aduh95]

I think ideally npm versions should not be coupled with Node.js at all, that would set npm free from doing releases however they please, and Node.js could release LTS versions without having to worry what npm is planning. At some point, we need to ask ourselves if the benefits of bundling a precise version of npm outweigh the downsides. Also, it gives npm an unfair advantage over other package manager clients, and maintains confusion between the Node.js project and the npm one.

I personally would like to see the TSC decide to wind down the bundling of npm with Node.js releases, as to me it appears as the most reasonable and fairest solution.

[kapouer]

The open source point of view totally seconds that. Having npm bundled in nodejs never made sense (from that point of view).

[MylesBorins]

Summary of npm9 Breaking Changes

Command Removals

bin

This command is not accurate, and having it available encourages users to rely on an incomplete implementation.

set-script

This has been deprecated in favor of the npm pkg set command.
birthday

birthday

This is an easter egg and has been flagged by automated malware tooling.

Breaking Changes to Commands

adduser

This command will no longer be an alias for login.

Config Removals

Bare _auth, _authToken, username, password, email

Bare configs like _auth are confusing and dangerous and will be removed. Instead they will be scoped to a specific registry.

sso-type and auth-type (except for legacy and webauthn)

This removes support for deprecated auth types.

loglevel=timing

This loglevel will be removed and instead timing info will be displayed only with --timing only which will decouple logging from generating timing information.

local, global-style, legacy-bundling

These configs will be replaced with a consolidated --strategy config with the above as values.
Default Config Changes
lockfile-version=3
npm install will create a v3 lockfile by default when creating a new lockfile. These lockfiles are incompatible with npm6.

auth-type=web

Make auth-type=web the default for login and publish for the public registry only.

install-links=true

npm install will install linked dependencies by default

init-private=true

npm init will create a package.json with private: true set by default.

access=public

npm publish will default to access=public for all packages. This is currently the default for non-scoped packages, so this change will now apply to scoped packages as well.

HEAD instead of master for opening links

Commands like npm docs will default to HEAD instead of master as the default ref for opening links.

Other Changes

Error when setting deprecated configs

npm9 will not run npm config set when key is a deprecated config item.

packlist ignore patterns

Two changes here:

Remove shrinkwrap.json as an unignorable file.

npm9 will standardize and document the order of operations of ignoring files from any command that produces a tarball. This is a subtle change but is being treated as breaking change since it will affect the files included in a tarball when multiple ignore types are used. npm/npm-packlist#88 has a full explanation of the specifics of this change.

Folder Permissions

npm will no longer attempt to set folder permissions. This is a complex issue and npm/rfcs#546 has the most up to date information about the specific changes.

Timing Log Files

With timing: true npm will write a process specific timing file similar to how it treats log files instead of appending a newline delimited json file.

engines: ^14.17.0 || ^16.0.0 || >=18.0.0

npm9 will set engines to the above. npm9 will also start to use node@14 specific features and fail when run on npm@12.

stdout and stderr Output

npm9 will standardize output between stdout and stderr. Errors will now be output on stdout when they can possibly be used programmatically by consumers.

Remove Progress Bar

npm9 will remove the progress bar to make it less error prone to receive user input during long running commands. This is not strictly a breaking change since npm does not consider display features to be part of its semver contract, but this change is being made now since it will be a visual change.

Normalize x, x@, & x@*

These specs are all the same but treated and resolved slightly differently. In npm9 they will all be resolved the same. This is a subtle change and is being included in npm9 in order to err on the side of caution. See npm/npm-package-arg#45 for the specific changes in the resolved values.

Remove path support from explain

Only allow for npm explain <package> instead of paths

[MylesBorins]

I think it would make sense to have a separate conversation if we want to talk about unbundling npm from Node.js .

We have to think about the usability concerns of permanently decoupling the two, and I think it would be good to discuss independently

[MylesBorins]

Quick ping on this to @nodejs/releasers and @nodejs/tsc

[BethGriggs]

Would landing npm@9 in Node.js 19, waiting a couple of months for feedback, and then backporting the update to Node.js 18 during LTS be an alternative option?

I personally think having a good amount of baking time for us to asses impact would be preferable to us rushing to land it in advance of the LTS transition the same month. (As we'll be breaking semver rules anyway, maybe we should do it once we've had more feedback from Node.js 19/Current so we can advise of any anticipated impact?)

[MylesBorins]

Waiting to update is definitely a reasonable solution, I'd still like us to review the proposed breaking changes and flag which changes are going to be "deal breakers" in advance.

[ruyadorno]

I'm a +1 from the point of view of a releaser. It's going to make it simpler to manage the v18.x release line moving forward, given that there's always extra work involved when managing multiple release lines of npm.

Personally, what I'm more concerned about is limiting the impact of breaking changes. With that in mind the more we can provide configurations that allows for users to opt-in to backwards compatibility the better. Would the npm team be willing to compromise on some of these intended changes (I believe removing bin and tweaking adduser cmds might be somewhat disruptive) in order to reduce any risk involved?