Yarn 2.0 (Yarn Plug’n’Play)

[jakeNiemiec]

Summary from whitepaper:
Pros:

• smaller install
• faster install
• better matches how gems behave
• better install & cache for CIs
• better peer dependencies
• users working on multiple projects across a system won’t pay in-creasing install costs
• no node_modules folder

Cons:

• no node_modules folder (some webpacker stuff currently relies on this). You also can't edit anything in node_modules
• packages must be accessed via yarn cache
• packages cannot rely on cross package boundaries (e.g.: webpacker users can't rely on the sub-dependencies of @rails/webpacker as if they were direct dependencies)

Resources:

• overview
• getting-started
• troubleshooting
• whitepaper
• webpack plugin

Newer links:

• https://dev.to/arcanis/introducing-yarn-2-4eh1
• https://next.yarnpkg.com/getting-started
• https://next.yarnpkg.com/advanced/migration

[omnilord]

Will pnp put all the modules in a singular global source of truth location on the local system, or will this still require downloading at least one new copy of them from the internet at every install, even if another project already uses the same version?

[jakeNiemiec]

From the whitepaper (very short read):

In its current state, running yarn install does the following under the hood -assuming a cold setup:

1. Dependency ranges are resolved into pinned versions
2. Packages for each version are fetched from their sources and stored in the offline mirror
3. The offline mirror is unpacked into the cache
4. The cache is copied into thenode_modulesfolders

Our solution aims to trim the fourth step from the equation. Instead of copying each and every file from the cache to the node_modules folder (which can take a lot of time, amplified by the sheer number of files), we will instead generate a single file that will contain static resolutions tables that list:

• What packages are available in the dependency tree
• How they are linked together
• Where they are located on the disk

So, instead of 100M of modules in a node_modules folder, that 100M is cached and you get a file that acts as a map to the modules that you need. No more filesystem traversal.

[paracycle]

PnP was exactly what I thought about when I read the thread at rails/rails#36282, great to see there is a will to tackle this.

I would be interested in taking a look at implementing this but I am super new to webpacker. Let me see how fast I can get familiar with it to make a contribution.

[jakeNiemiec]

I have tried it out and can confirm that webpacker needs to implement something like peerDependencies in order to make this work. Error with the default setup: You cannot require a package ("@babel/preset-env") that is not declared in your dependencies:

Add this to package.json to enable.

"installConfig": {
    "pnp": true
  }

The errors

Uncaught Error: Module build failed (from /Users/user/Library/Caches/Yarn/v4/npm-babel-loader-8.0.6-e33bdb6f362b03f4bb141a0c21ab87c501b70dfb/node_modules/babel-loader/lib/index.js):
Error: You cannot require a package ("@babel/preset-env") that is not declared in your dependencies (via "/Users/user/webpacker_tests/hello/babel.config.js")
    at makeError (Users/user/webpacker_tests/hello/.pnp.js:55)
    at Object.resolveToUnqualified (Users/user/webpacker_tests/hello/.pnp.js:11126)
    at Object.resolveRequest (Users/user/webpacker_tests/hello/.pnp.js:11204)
    at Function.Module._resolveFilename (Users/user/webpacker_tests/hello/.pnp.js:11386)
    at Function.Module._load (Users/user/webpacker_tests/hello/.pnp.js:11302)
    at Module.require (internal/modules/cjs/loader.js:637)
    ...

If you meticulously install all of the deps you need with PnP, the starter files will actually compile successfully:

@gauravtiwari, This is only possible since you already did some work last year to enable PnP 🎉: https://github.com/rails/webpacker/pull/1823/files#diff-1701de23f718ce36149c9b23e2db08caR95

First glance action items:

• Fix check_yarn_integrity to do something like this PR
• Find a way to hoist deps to the users "top" package.json. Here is an example that of another CLA tool doing this
  • Figure out which deps need to get hoisted from package.json. This would include everything require()'d in package/environments/base.js
• Should PnP be the default? No, but there should be an option to enable.
• require("channels") needs to be changed to require("../channels"). Otherwise it throws an error because it looks for a package called channels.

[gauravtiwari]

Thanks @jakeNiemiec Sorry about the delay. I will take a look tomorrow and get back to you.

[somebody32]

just my 2 cents on Should PnP be the default?: https://twitter.com/sebmck/status/1137059802838757377?s=20. Obviously, Facebook's scale is a different story, but anyway

[jakeNiemiec]

Yarn 2.0 summary post

With yarn@2 released, I assume that other users will attempt to upgrade and use with webpacker. This will not work since yarn now enforces package boundaries.