Suggestion for new lint: Cognitive Complexity

[felix91gr]

Hi all, I'd like to propose (and implement, if someone can mentor me a bit!) a lint for Cognitive Complexity. It's a way to measure the cognitive load that a particular function puts on the reader.

I think that having an upper bound on the allowed cognitive complexity of functions would help ensure that complex functions are split into simpler, more maintainable ones. It would also help to complement the role that the Cyclomatic Complexity lint currently has.

[oli-obk]

Our cyclomatic complexity lint already adresses a few of the issues mentioned in the cognitive complexity paper. E.g. a match has a CC of 2 regardless of the number of arms.

I wonder if it would make sense to simply rename our cyclomatic complexity lint to cognitive complexity and start integrating a few more of the suggestions from the linked paper

[Manishearth]

Yeah I think that would be better.

[felix91gr]

Aside: let's call Cyclomatic Complexity as "CyC" and Cognitive Complexity "CoC"? Since both of them can be called CC, but are not the same thing x3

E.g. a match has a CC of 2 regardless of the number of arms.

Here I assume you meant CyC. If so... does the CyC lint in Clippy give all match's a score of 2? Shouldn't it depend on the number of branches of the match, since CyC measures...

"(...) the number of linearly independent paths through a program's source code."

(Quote from Wikipedia's first paragraph on CyC.)

I wonder if it would make sense to simply rename our cyclomatic complexity lint to cognitive complexity and start integrating a few more of the suggestions from the linked paper.

In general, for introduction purposes, I think it makes sense! You'd have the same lints as always, but the current CyC lint becomes more powerful. Users just get an upgrade of their already great tool.

But in the long term I wonder if it might be better to keep them separate. CyC is better as a metric of how hard a piece of code is to test, while CoC is better as a metric of how hard a piece of code is to understand. (Code Climate's article comparing both is very good to illustrate this point)

Also I think we'd prefer (please correct me if I'm wrong!) to keep the lints smaller, because it would allow a user to use Clippy to tackle smaller, more approachable problems.

[Manishearth]

Shouldn't it depend on the number of branches of the match,

Right, @oli-obk's point is that our CyC lint doesn't actually measure CyC, it measures something in between CyC and CoC, which is what folks found to be more useful. Moving it all the way towards CoC is fine by me. As a linter engine "hard to test" is far less useful and actionable advice to give.

…

On Thu, Feb 21, 2019, 11:21 PM Félix Fischer ***@***.***> wrote: *Aside:* let's call Cyclomatic Complexity as "CyC" and Cognitive Complexity "CoC"? Since both of them can be called CC, but are not the same thing x3 E.g. a match has a CC of 2 regardless of the number of arms. Here I assume you meant CyC. If so... does the CyC lint in Clippy give all match's a score of 2? Shouldn't it depend on the number of branches of the match, since CyC measures... "(...) the number of linearly independent paths through a program's source code." (Quote from Wikipedia <https://en.wikipedia.org/wiki/Cyclomatic_complexity>'s first paragraph on CyC.) I wonder if it would make sense to simply rename our cyclomatic complexity lint to cognitive complexity and start integrating a few more of the suggestions from the linked paper. In general, for introduction purposes, I think it makes sense! You'd have the same lints as always, but the current CyC lint becomes more powerful. Users just get an upgrade of their already great tool. But in the long term I wonder if it might be better to keep them separate. CyC is better as a metric of how hard a piece of code is to test, while CoC is better as a metric of how hard a piece of code is to understand. (Code Climate's article <https://docs.codeclimate.com/docs/cognitive-complexity> comparing both is very good to illustrate this point) Also I think we'd prefer (please correct me if I'm wrong!) to keep the lints smaller, because it would allow a user to use Clippy to tackle smaller, more approachable problems. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#3793 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABivSIv5yFX2t1wDbBeZh-n0tFCFsOasks5vPtypgaJpZM4bGkvY> .

[felix91gr]

it measures something in between CyC and CoC, which is what folks found to be more useful.

Ohhhhhhhhhhhhh, this makes much more sense to me now.

Moving it all the way towards CoC is fine by me. As a linter engine "hard to test" is far less useful and actionable advice to give.

If there's consensus on this, I'm willing to help :)

[oli-obk]

Yea, pure CyC has been found to be pretty useless in Rust as you often use matches not to run different code but more as a lookup table for values.

Yea, I think we can just go ahead with this. As a first step I would nuke the cfg function call and the math resulting from it and just keep the numbers from the visitor. Then, change the visitor to match the CoC rules. If I understood the paper correctly, it's all syntactical, so we should probably move to a pre-expansion-pass. IMO macro calls reduce complexity, even if macro definitions increase it

[oli-obk]

Oh, btw, feel free to bug me about this here, on discord or on irc. I'm very interested in any musings you have on this.

[felix91gr]

With Oliver (@oli-obk) we're now working on this. If anyone wants to chip in, reach us ^^ we plan on making an MVP first and leave the space for later expansions on more fine-grained or complex scenarios, as the use-cases start to appear.

[felix91gr]

Draft of CoC implementation

Input

A Rust function.
(Option for the future: a Rust module, aggregating the CoC of its functions)

Output

A Cognitive Complexity score that loosely fits the Specification laid out in the paper.

How it works

From now on, (I can change it later if people prefer I copy some of the paper here!) this text assumes the reader is familiar with the paper.

Nesting Structures

The following structures increment the nesting level:

• if, else, else if
• if let
• match
• for, while, loop
• Nested functions
• Closures

(Nesting-Dependent) Increments

For each of the following constructs, there is an increment in the CoC score of (1 + nesting level).

• if
• match
• for, while, loop

(Nesting-Independent) Increments

For each of the following constructs, there is a constant increment of 1 in the CoC score.

• break, continue
• Changes in sequences of binary logical operators (these 5: &&, ||, &, |, ^)
• Recursive function calls
• Macro calls

Exempt of Increment

The following constructs are explicitly exempt of increment, since they replace and simplify other constructs.

• ? operator for error propagation
• if let statement

Pending Points

Here are things that we're explicitly postponing until we have the tools or the know-how necessary to implement them.

• What to do about conditional compilation? Currently Clippy can't reach that information, but things like C's #if should at least add +1 to the nesting level.

---

I need help with Rust specific features I might be missing. I'm still relatively new to the language, so any help is appreciated! 🙏

Changelog

• 16/04: clarified some points regarding the Binary Logical Operators
• 02/04: added Macro Calls to the list of (Nesting-Independent) Increments constructs. See Ignore cognitive complexity in macro expanded code #3900
• 21/02: added Pending Points section, and moved Conditional Compilation to it.
• 21/02: removed Ternary Operator, for it does not exist in Rust.
• 21/02: added Exempt of Increment subsection, with 2 cases.
• 21/02: first draft

[oli-obk]

* Ternary Operator (`a = q ? b : c;`)

doesn't exist in Rust ;)

* Nested functions

Are these a problem? How is it less complex to have the function outside?

* Is there conditional compilation in Rust? That should increment the nesting level as well.

we have cfg attributes, but clippy doesn't see them right now. There are a bunch of things we want to do with them, so ignore this until we figure out how to get them.

* Sequences of binary logical operators

both short circuiting and non-short-circuiting are relevant here imo

Also, if I read the paper correctly, only changes of the binary operator in a sequence cause an increment. So x && y && z is fine, but x && y || z will increment

I need help with Rust specific features I might be missing. I'm still relatively new to the language, so any help is appreciated! pray

Since everything is an expression, I think having any nesting expression inside e.g. a binary operation (x + if cond { y } else { z }) or a function call (foo(x, y, match bar { X => 42, _ => 0 })) should also cause increments.

break from a loop seems fairly nonproblematic. Not sure if that should increase the CoC even further than the loop by itself did.

[felix91gr]

doesn't exist in Rust ;)

Haha 😆 my bad, I'm definitely still learning.

• Nested functions

Are these a problem? How is it less complex to have the function outside?

In the paper, nested functions are considered part of the body of the function itself. They add no increment, but they are like a new if-block in that they add nesting. I think that's because they are being treated as closures, capturing the surrounding context.

Nested functions in Rust can't do that right? For all intents and purposes, they might as well be an external function, but that can only be seen from the scope of the containing fuction. Right? If so, I think we could for the most part treat nested functions as an external function entirely, having its own score and not affecting the containing function's score at all.
(Edit: someone already told me the answer to the capturing questions)

we have cfg attributes, but clippy doesn't see them right now. There are a bunch of things we want to do with them, so ignore this until we figure out how to get them.

Oki doke!

both short circuiting and non-short-circuiting are relevant here imo

Same.

Also, if I read the paper correctly, only changes of the binary operator in a sequence cause an increment. So x && y && z is fine, but x && y || z will increment

Almost. x gives +0, x && y gives +1 and so does x && y && z. x && y || z gives +2. It's +1 as soon as a binary operator is introduced, and +1 for every time it changes.

Since everything is an expression, I think having any nesting expression inside e.g. a binary operation or a function call should also cause increments.

Good point! We do have to consider this from the perspective of expressions. I think your examples are right. Those expressions introduce abstraction, and that should be accounted for :)

break from a loop seems fairly nonproblematic. Not sure if that should increase the CoC even further than the loop by itself did.

That's a fair point. Hmm. Since there can be many breaks inside a loop, maybe the loop should add just 0, putting its cognitive cost mostly in the breaks that stop it 🤔