RecursionError: maximum recursion depth exceeded while linting a large chained method calls

[yilei]

Bug description

When parsing the following file:

from a import b

(
    b.builder('name')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .add('name', value='value')
    .Build()
)

Configuration

No response

Command used

pylint t.py

Pylint output

Traceback (most recent call last):
  File "pylint/lint/pylinter.py", line 1044, in get_ast
    return MANAGER.ast_from_file(filepath, modname, source=True)
  File "astroid/manager.py", line 124, in ast_from_file
    return AstroidBuilder(self).file_build(filepath, modname)
  File "astroid/builder.py", line 145, in file_build
    return self._post_build(module, builder, encoding)
  File "astroid/builder.py", line 173, in _post_build
    module = self._manager.visit_transforms(module)
  File "astroid/manager.py", line 95, in visit_transforms
    return self._transform.visit(node)
  File "astroid/transforms.py", line 89, in visit
    return self._visit(module)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 61, in _visit_generic
    return [self._visit_generic(child) for child in node]
  File "astroid/transforms.py", line 61, in <listcomp>
    return [self._visit_generic(child) for child in node]
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)

  <.... SKIPPED ....>

  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 54, in _visit
    visited = self._visit_generic(value)
  File "astroid/transforms.py", line 67, in _visit_generic
    return self._visit(node)
  File "astroid/transforms.py", line 57, in _visit
    return self._transform(node)
  File "astroid/transforms.py", line 38, in _transform
    if predicate is None or predicate(node):
  File "astroid/brain/brain_builtin_inference.py", line 141, in _builtin_filter_predicate
    and node.root().name == "re"
  File "astroid/nodes/node_ng.py", line 371, in root
    return self.parent.root()
  File "astroid/nodes/node_ng.py", line 371, in root
    return self.parent.root()
  File "astroid/nodes/node_ng.py", line 371, in root
    return self.parent.root()
  [Previous line repeated 319 more times]
RecursionError: maximum recursion depth exceeded

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "pylint/lint/pylinter.py", line 730, in _get_asts
    ast_per_fileitem[fileitem] = self.get_ast(
  File "pylint/lint/pylinter.py", line 1066, in get_ast
    raise astroid.AstroidBuildingError(
astroid.exceptions.AstroidBuildingError: Building error when trying to create ast representation of module 't'

Expected behavior

It should not crash.

Pylint version

pylint 2.17.4
astroid 2.15.5
Python 3.10.8 (main, Mar  9 2023, 10:11:52) [GCC 12.2.0]

OS / Environment

No response

Additional dependencies

No response

[DanielNoord]

I never know with these kind of issues. pylint probably shouldn't crash but we also shouldn't try to be too sensical about it. I think it is fine to notify the user that their code is unparseable, after all we are a linter.. 😄

Curious to see what others think.

[yilei]

Ideally, there should be a way to tell the linter, hey, please ignore this line (pylint: disable=), so it can still check the rest of the file.

At minimum, # pylint: skip-file should work.

[Pierre-Sassoulas]

I think we decided at some point that explicitely failing is better than silentely failing the inference and having a poorer result without any explanation. At least here it's possible to increase the recursion limit. Skipping the parsing conditionally will be an enormous feature. Right now skip file mean skipping the linting so it would also need a new concept (like '# astroid: no-inference')

[Pierre-Sassoulas]

Thought about it some more, I think we need to transform all crashes into fatal level messages. There's no reason to prevent the remainder of the analysis to take place when there's a crash, and it's still possible to report the crash when it happens (the message being essentially the same but instead of crashing pylint continue working if possible). That way it's possible to ignore the fatal with the disable when they are expected (until the crash is fixed in pylint). I already had some project were pylint was unusable because of a crash related to pandas, nothing to do but to wait for a fix, it's annoying.

[jacobtylerwalls]

There are several places where we could try to trap RecursionError and reraise it as a fatal (but ignorable) message.

1. Inference system
2. Performing astroid transforms (this issue)

[jacobtylerwalls]

pylance doesn't crash, but it does admit it can't parse. We can do something similar.

Maximum parse depth exceeded; break expression into smaller sub-expressions: Pylance