Plans for npm 10

[lukekarrys]

The npm team is working on npm 10, and we are currently in the planning and early development phase. In opening this issue our goal is to facilitate discussion about our planned breaking changes in npm 10 and work towards our goal of landing npm 10 in Node.js 20 and 18.

Breaking Changes

During the discussion about npm 9 we landed on a list of guidelines on integrating new major versions of npm with Node. We plan on following these guidelines again, and we consider all breaking changes listed below to fall within those guidelines.

• Update engines to >=16.14 ^18.17.0 || >=20.5.0
  • Tracking issue: Engines setting for npm 10 and beyond npm/statusboard#640
• Remove implicit if-present logic from run-script in workspace mode
  • Currently when calling npm run-script on workspaces if any of the scripts are missing it is treated as a silently ignored error, making there an implicit "if-present" config in this mode. Users who want this behavior will need to include this config.
• Remove the following deprecated or unused configs: tmp, ci-name, hashAlgorithm, metrics-registry
  • All these configs are either deprecated or are entirely unused in the npm CLI. We see these changes as low impact, but we always treat the removal of a config item as a breaking change.
• Use @npmcli/agent
  • The existing http agent and proxy libraries we use do not have very good timeout configurations and also diverge from the spec in odd ways. The new agent is intended to be identical to the existing way of doing things, but because of the scope of the change we are waiting on a semver major before cutting over.
• Remove "strict mode" from npm-package-arg
  • npm-package-arg can be set to run in a RFC 8909 compliant "strict mode" by setting the environment variable NPM_PACKAGE_ARG_8909_STRICT=1. We don't have any real plan with making that mode permanent so we are removing this functionality. Too many existing packages rely on our technically spec divergent behavior and it is not worth breaking so many people to force compliance. Anyone currently using this mode will need to find a third party way to strictly enforce the package specs in their package.json.
• Don't retry 409 errors during npm publish
  • Currently if the npm CLI gets a 409 from the registry during publish it attempts a single retry by re-fetching the manifest and attempting to apply a new patch on what's being published. This is not the safest approach, especially as the code used to patch is not the same code that is used to build the original manifest. It is much safer to simply have the user re-run the publish command.

Note: the following changes were originally planned but are not a part of npm@10

• Expand the list of default ignored files obeyed during pack and publish
  • Tracking issue: [FEATURE] Expand the list of default ignored files npm/npm-packlist#48
  • RFC issue: RFC: Expand list of ignored files npm/rfcs#114
  • The exact list of files is still being discussed, but npm will start ignoring more files by default from being included in packaged tarballs. The npm CLI currently defines this list in npm-packlist and it includes files like .npmignore and .gitignore. The original RFC proposes adding files to that list such as .editorconfig, .github/, and more.
• Implement the "obey user specifier" RFC
  • RFC issue: add "obey user specifier" RFC npm/rfcs#547
  • The npm CLI will now save the same specifier used to install a package to package.json, eg npm install eslint@=8.8.0 will save "8.8.0" as the specifier for eslint instead of "^8.8.0" as it does now. This will only affect installs run with an argument, and bare npm install commands will not churn the lockfile or package.json.

Testing

citgm was very helpful during the release of npm 9 in finding unintended consequences of our breaking changes. One new thing we will be doing during this release cycle is running our own modified version of citgm in GitHub Actions during all prereleases candidates of npm 10 to ensure we find breaking change bugs as early as possible.

Testing will also happen in CI during each pull request that is opened by the npm team.

Timeline

Our goals with this timeline are to ensure that npm 10 changes can be tested within Node as early as possible. We want to open PRs to main so that CI can be run against each prerelease of npm 10. The initial PRs will be labeled to only request-ci and not ready to be merged.

Once npm 10 is stable (Task 4) we will open a new PR that will be ready for review and to land on main. Once that PR is merged we will open PRs for both Node 20 and Node 18 backports. Our goal with these PRs is again to allow testing to happen as early as possible.

For Node 20 our goal is that npm 10 could be included in the tentative 2023-09-12 v20.10.0 release. For Node 18, assuming that npm 10 has shipped to users and we've fixed any bugs that have been raised, our goal is that npm 10 could be included in the tentative 2023-10-xx v18.20.0 release.

	Status	Task	Date
1	✅	First prerelease (npm@10.0.0-pre.0)	2023-07-19 2023-07-26
2	✅	Open initial PR* to Node.js main branch for CI only nodejs/node#48934	2023-07-19 2023-07-26
3	✅	Second prerelease	2023-08-02 2023-08-31
4	✅	Major release (npm@10.0.0)	2023-08-16 2023-08-31
5	✅	Open PR to Node.js main branch for review nodejs/node#49423	2023-08-16 2023-08-31
6	✅	Land npm 10 on Node.js main branch	On or before 2023-08-23 2023-08-31
7	✅	Promote npm@10.0.0 to latest	2023-08-30 2023-09-08
8		Open PR to backport to Node.js 20	After npm 10 lands on main (Task 6)
9	✅	Open PR requesting backport to Node.js 18 nodejs/node#49611	After npm 10 lands on main (Task 6) 2023-09-11
10		Release npm 10 in Node.js 20	2023-09-12** v20.7.0
11		Release npm 10 in Node.js 18	2023-10-xx** v18.20.0

* Unmerged pull requests will be closed and a new one opened for each (pre)release. Any new PR will contain all the changes from all previous unmerged PRs.

** Tentative Node.js release date

[mcollina]

The plan looks solid and I don’t think there would be issue in shipping npm v10 in Node 20. However, I feel that shipping such an update to Node v18 might be too risky for two reasons:

1. Node.js v16 is out of LTS at the end of September
2. Node.js v18 enters maintenance soon after
3. Node.js v20 goes LTS end of October

I don’t think there is much time to iron out possible bugs with thr given timeline.

Lastly, I recommend the npm team to stop treating the LTS release as their target and align their releases to Node.js. We could have avoided this discussion entirely if you released npm v10 in March or April.

[ruyadorno]

Expand the list of default ignored files obeyed during pack and publish

From the list of breaking changes I believe this is the most likely to have a potential to be disruptive to the community, if the goal is to land on LTS I would advise the team to be very defensive about expanding that list. It might be worth holding on to the next major which hopefully aligns better with our release calendar.

[wraithgar]

from the list of breaking changes I believe this is the most likely to have a potential to be disruptive to the community, if the goal is to land on LTS I would advise the team to be very defensive about expanding that list. It might be worth holding on to the next major which hopefully aligns better with our release calendar.

I agree and this is really the suggestion that moved the needle back to "let's not do this". We were already on the fence on this one and the cost/benefit is not anywhere near the level that we would want on a breaking change.

[lukekarrys]

npm@10 has been released and promoted to latest. I updated my original comment which some changes as to what breaking changes actually landed in npm@10 as well as updating the dates to when those events happened.

The PR to land npm@10 in nodejs/node#main has been opened here and is ready for review: nodejs/node#49423

[lukekarrys]

I have opened a PR requesting to backport npm@10 to Node 18 here: nodejs/node#49611. This PR is just to run CI and facilitate discussion, since I expect opening the PR to generate more 👀 from contributors. npm@10 should be landing in Node 20 soon and we can continue to get and respond to user feedback.

We've already triaged and fixed one regression (npm/cli#6760) from npm 9 to 10 and are continuing to monitor new issues coming in for npm 10. At this time we have no new npm P0 or P1 bugs in npm@10.

Based on the list of breaking changes that ended up in npm@10 (2 of the more potentially disruptive changes were dropped from the release), we think it makes sense to land npm@10 in Node 18 before LTS. The smaller set of ubreaking changes means the potential surface area for user disruption is smaller than then the process of updating npm 8 to 9.

Lastly, I recommend the npm team to stop treating the LTS release as their target and align their releases to Node.js. We could have avoided this discussion entirely if you released npm v10 in March or April.

Addressing this point directly, our future plans will take this into account. Going forward we want to align more closely with Node around major versions to make this process much smoother. But we think in the case an exception for npm 10 should be discussed.

[mcollina]

It would be best to avoid shipping npm 10 in a release, given that glob now depends on BlueOak-1.0.0 licensed software, which is against the IP policy of the foundation. I've already asked an opinion to the Foundation legal staff.

I'm acting out of caution here.

[bricss]

It looks like BlueOak-1.0.0 is very much state-of-the-art permissive license, that is shipped within npm v9 in Node already.
And here is good article about it.

N.B.: Trying a new permissive software license

[mcollina]

Ref nodejs/node#49625.