Extracting the tar content of undefined failed

[opiation]

Do you want to request a feature or report a bug?
Reporting a bug when running yarn install to install node dependencies. For severity, this bug seems critical considering it essentially prevents me from acquiring node dependencies.

What is the current behavior?
Fails sometimes with errors like the following:

yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.10.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v2/npm-lodash-4.17.10-1b7793cf7259ea38fb3661d4d38b3260af8ae4e7/_cacheHas.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/lodash/-/lodash-4.17.10.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "EEXIST: file already exists, mkdir '/usr/local/share/.cache/yarn/v2/npm-lodash-4.17.10-1b7793cf7259ea38fb3661d4d38b3260af8ae4e7'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.

yarn install v1.9.4
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
[1/4] Resolving packages...
[2/4] Fetching packages...
error https://registry.yarnpkg.com/fbjs/-/fbjs-0.8.17.tgz: Extracting tar content of undefined failed, the file appears to be corrupt: "ENOENT: no such file or directory, chmod '/usr/local/share/.cache/yarn/v2/npm-fbjs-0.8.17-c4d598ead6949112653d6588b01a5cdcd9f90fdd/lib/resolveImmediate.js'"
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command

The occurrence of this error is the challenging part. It does not always fail and does not always fail with the same dependency. The installation is sometimes successful after 3-5 tries.

If the current behavior is a bug, please provide the steps to reproduce.
I've attempted to install dependencies on bare-metal and in a node:8-alpine docker container. Both can sometimes encounter the error. I've tested this on my personal device in Montreal, Canada (Mac OS X10.13), on a AWS EC2 instance (Ubuntu 18.04), on a GCE instance (Ubuntu 16.04) and on a production server in France (Debian 8). Each of them can sometimes encounter this error. I've also tried installing with and without yarn.lock to no avail. Find a package.json that appears to sometimes reproduce the issue in this gist. The issue does not seem to happen with projects having fewer dependencies.

What is the expected behavior?
Successful installation of all packages like npm install or npm ci which deterministically succeeds without any tar or caching errors.

Please mention your node.js, yarn and operating system version.
Tested with the following version:
Node: 8 LTS, 10
Yarn: 1.9.2, 1.9.4
OS: Ubuntu 18.04 LTS, Ubuntu 16.04 LTS, Debian 8, Mac OSX 10.13
Registrie: registry.yarnpkg.com, registry.npmjs.org, private registry

If you require any additional information, don't hesitate to request it. FWIW, reducing the network-concurrency seems to produce a slightly higher success ratio but not consistently enough to conclude the errors are related. It may be an area to investigate however. I've unfortunately exhausted all the time I could afford to spend on this after a few days of troubleshooting. I've reluctantly had to migrate all our CI builds back to using npm install/npm ci :(

[Titozzz]

Same issue, it's blocking my CI too, we updated recently to yarn 1.9.2

[adrienharnay]

@opiation The error is indeed random but I may have found the cause: do you have distant git URLs in your package.json without .git at the end? We had two of those and adding .git fixed the issue. Not sure why the error message doesn't directly state this is is the problem though.

[adrienharnay]

Also, maybe related: #5536

[opiation]

@adrienharnay, can you define what you mean by distant? For the record, here's the package.json I used. There's only one github dependency and I still get errors without it. I'm not sure how I would append .git to non-git dependencies, unless I've misunderstood your suggestion.

[adrienharnay]

Distant wasn't the right word, I just meant packages installed from Git 🙂

Could you try this?

"storybook-addon-markdown": "https://github.com/mihalik/storybook-addon-markdown.git"

[opiation]

As per my previous comment, I still encounter the issue without storybook-addon-markdown dependency. Thus, I don't believe this issue stems from yarn improperly handling git URLs.

[adrienharnay]

Indeed, I read too quickly. Well, that fixed our bug, but I have no idea about yours 😕 Sorry

[Titozzz]

@opiation Did you update you yarn.lock file too? Cause I had to do that

[opiation]

@Titozzz, I encounter this error with and without the yarn.lock file. I have deleted and recreated the lock file several times to no avail.

[ekarlso]

I get this too and I dont have any packages from git.

[luwes]

I wanted to get around this issue (#6256) by using the tarball versions of the packages but indeed the error above is thrown for tarball urls on self-hosted Github enterprise.

github.com hosted tarballs somehow did work. e.g.
https://github.com/luwes/chameleon/archive/grasshopper-v0.0.1-beta.4.tar.gz

[kvnify]

I'm seeing the same issue with a project we have. However when I remove the deps that run a prepare script as part of the install (due to being git urls) then it works. These happen to be pointing git urls, but I think it's actually the prepare that seem to kick off more yarn install processes that seem to subvert the mutex flag for some reason. I wonder if that's because the other processes are kicked off by the root process rather than by different root processes. I don't know if this information helps or if its actually way off base. But I figured I would share what I've found regardless.

[adrienharnay]

@Khendry I got the issue again, and you are right, it comes from git dependencies which have a prepare script in their package.json! 👍