"Maximum call stack size exceeded" in streamChunks*

[WolfspiritM]

Using storybook with webpack5 we recently ended up getting "Maximum call stack size exceeded" when we try to build the storybook project.

    RangeError: Maximum call stack size exceeded
    at String.match (<anonymous>)
    at streamChunksOfRawSource (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfRawSource.js:14:25)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfRawSource.js:40:5)
    at RawSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\RawSource.js:56:10)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at streamAndGetSourceAndMap (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53)
    at CachedSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\CachedSource.js:208:35)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ReplaceSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\ReplaceSource.js:176:44)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ConcatSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\ConcatSource.js:112:55)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at streamAndGetSourceAndMap (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamAndGetSourceAndMap.js:27:53)
    at CachedSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\CachedSource.js:208:35)
    at module.exports (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunks.js:13:17)
    at ConcatSource.streamChunks (C:\src\node_modules\webpack\node_modules\webpack-sources\lib\ConcatSource.js:112:55)

It seems to be caused by the regex used for the match. I have seen it in two different spots. The other one was:

UnhandledPromiseRejectionWarning: RangeError: Maximum call stack size exceeded
    at String.match (<anonymous>)
    at streamChunksOfSourceMapLinesFull (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:167:23)
    at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:405:6)
    at nameIndexMapping.<computed> (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfCombinedSourceMap.js:271:5)
    at streamChunksOfSourceMapLinesFinal (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:331:3)
    at module.exports (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfSourceMap.js:398:6)
    at streamChunksOfCombinedSourceMap (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\streamChunksOfCombinedSourceMap.js:54:9)
    at SourceMapSource.streamChunks (\node_modules\webpack\node_modules\webpack-sources\lib\SourceMapSource.js:197:11)
    at exports.getSourceAndMap (\node_modules\webpack\node_modules\webpack-sources\lib\helpers\getFromStreamChunks.js:17:33)
    at SourceMapSource.sourceAndMap (\node_modules\webpack\node_modules\webpack-sources\lib\SourceMapSource.js:188:10)
    at getTaskForFile (\node_modules\webpack\lib\SourceMapDevToolPlugin.js:84:30)
    at \node_modules\webpack\lib\SourceMapDevToolPlugin.js:272:22
    at \node_modules\webpack\lib\Cache.js:93:5
    at Hook.eval [as callAsync] (eval at create (\node_modules\webpack\node_modules\tapable\lib\HookCodeFactory.js:33:10), <anonymous>:6:1)
    at Cache.get (\node_modules\webpack\lib\Cache.js:75:18)
    at ItemCacheFacade.get (\node_modules\webpack\lib\CacheFacade.js:117:15)

At first I thought it might be cause of a big chunk and source file but I tried to output "source" prior to the "source.match" and it seems to only contain:

/* module decorator */ module = __webpack_require__.hmd(module);\n

I'm not sure how to create a reproducable way for that issues however it vanishes when downgrading to webpack ~5.46.0 and whenever I upgrade back to ~5.47.0 the error reappears.

[alexander-akait]

Can you provide example how we can reproduce it?

[WolfspiritM]

I sadly can't produce a good example cause the source I'm using is private and it seems like the reason for that is really the size.

I tried to investigate further and it seems like the reason really is a too large "source".

I have a source that is has a length of 11846003 characters which causes that issue. As these streamChunk functions seem to be called asynchronously I just ended up having the small line as the last one before the crash. However the problem is with that large file (and if out of some reason that file works then it happens with the next one wich is twice as large). I exported that file and called new RawSource(filecontent) and then streamChunks manually but that didn't cause the issue.

In some rare cases it seems to work. I have no idea how to procede. I tried to make a reproducable example via a new repository but I don't have the error there.

I think the source is just too large for the regex (as it always happens with the String.match function) but I actually have no idea why it works when calling RawSource manually.

Any idea?

[alexander-akait]

How often does this happen? Do you build application in dev or production mode? Do you use webpack dev server when got this problem? Do you generate source maps? What is value of devtool?

[WolfspiritM]

How often does this happen?

About 9 out of 10 times. Very often. With 5.46.0 it never happened.

Do you build application in dev or production mode?

It happens when building the storybook static version. According to their docs it's building production mode.

Do you use webpack dev server when got this problem?

No. Also using storybook watch mode works fine.

Do you generate source maps? What is value of devtool?

Value of devtool is cheap-module-source-map (which seems to be the storybook default).
When setting devtool to undefined before the build the build succeeds all the time.

I assume there is a huge sourcemap that is causing that issue.

[alexander-akait]

Maybe you are right, can you try to debug values passed to at String.match?

[alexander-akait]

I am afraid it can be ddos regexp...

[WolfspiritM]

The regexp passed is the one from here:

webpack-sources/lib/helpers/streamChunksOfRawSource.js

Line 10 in 10d5dae

const MATCH_LINES_REGEX = /(?:[^\n]+\n?|\n)/g;

while the source string where the match is applied to is the 11846003 long string.
It starts with:
const __WEBPACK_NAMESPACE_OBJECT__ = JSON.parse('

And seems to be a very huge json with properties like name, type, description, methods, args. I guess that's a source map.

What I find interesting however is that I wrote that huge file to a textfile and tried a simple script to reproduce it by doing:

const t = require('webpack-sources');
const fs = require('fs');
const file = fs.readFileSync('test.js', "utf8");
var s = new t.RawSource(file);
s.streamChunks({}, () => {}, () => {}, () => {});

And that works without problems...so there must be something else in play. Either an already large call stack (but it doesn't seem to be that large) or maybe the parallel execution that causes it to trip?!

[alexander-akait]

Either an already large call stack (but it doesn't seem to be that large)

I think this... We need rewrite this, but it can't be very easy

[nukithelegend]

We're having the same issue in our Angular 12 project after updating our packages. I initially thought it had to do with SASS as picked up in other threads (https://stackoverflow.com/questions/68756057/maximum-call-stack-size-exceeded-in-streamchunksofsourcemap-js), however, this makes more sense.

This happened in multiple branches when updating different packages, so it makes sense that our project might have reached the size limit that causes this issue. I currently can't build our project at all.

Are there any short-term workarounds for this?

[alexander-akait]

No short-term workaround, we need to fix it here

[nukithelegend]

Downgrading Angular to 12.0.5 worked for me. Why would this resolve the issue though?

[alexander-akait]

Maybe you can provide steps to reproduce? We really need to investigate it

[nukithelegend]

Unfortunately it's a private repository and I won't be able to.

We can always try to use the angular stress-test project, it might be big enough. Work is a bit hectic right now, I'll try to set it up in a few days.

If anyone else wants to attempt it, I just know that updating to the latest Angular CLI version is what caused this. Reverting to 12.0.5 fixed it.

[Bjeaurn]

I think I have time today. I'm not familiar with the angular stress-test project but I'm sure I can find it. I'll take a stab at it and let you know.