-
Notifications
You must be signed in to change notification settings - Fork 2k
Fix incorrect location data in OUTDENT nodes #4291
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix incorrect location data in OUTDENT nodes #4291
Conversation
In f609036, a line was changed from `if length > 0 then (length - 1) else 0` to `Math.max 0, length - 1`. However, in some cases, the `length` variable can be `undefined`. The previous code would correctly compute `lastCharacter` as 0, but the new code would compute it as `NaN`. This would cause trouble later on: the end location would just be the end of the current chunk, which would be incorrect. Here's a specific case where the parser was behaving incorrectly: ``` a { b: -> return c d, if e f } g ``` The OUTDENT tokens after the `f` had an undefined length, so the `NaN` made it so the end location was at the end of the file. That meant that various nodes in the AST, like the `return` node, would incorrectly have an end location at the end of the file. To fix, I just reverted the change to that particular line.
|
Is it possible to write a test for this? |
|
Why do they have a undefined length? Shouldn't it be at least 0? |
|
@vendethiel To be honest I don't know this code that well, but my impression is that @lydell When I initially looked I thought there weren't any tests directly for the lexer, but now I see that |
|
LGTM |
Backport of jashkenas#4291 . In f609036, a line was changed from `if length > 0 then (length - 1) else 0` to `Math.max 0, length - 1`. However, in some cases, the `length` variable can be `undefined`. The previous code would correctly compute `lastCharacter` as 0, but the new code would compute it as `NaN`. This would cause trouble later on: the end location would just be the end of the current chunk, which would be incorrect. Here's a specific case where the parser was behaving incorrectly: ``` a { b: -> return c d, if e f } g ``` The OUTDENT tokens after the `f` had an undefined length, so the `NaN` made it so the end location was at the end of the file. That meant that various nodes in the AST, like the `return` node, would incorrectly have an end location at the end of the file. To fix, I just reverted the change to that particular line. Also add a test that tokens have locations that are in order
Backport of jashkenas#4291 . In f609036, a line was changed from `if length > 0 then (length - 1) else 0` to `Math.max 0, length - 1`. However, in some cases, the `length` variable can be `undefined`. The previous code would correctly compute `lastCharacter` as 0, but the new code would compute it as `NaN`. This would cause trouble later on: the end location would just be the end of the current chunk, which would be incorrect. Here's a specific case where the parser was behaving incorrectly: ``` a { b: -> return c d, if e f } g ``` The OUTDENT tokens after the `f` had an undefined length, so the `NaN` made it so the end location was at the end of the file. That meant that various nodes in the AST, like the `return` node, would incorrectly have an end location at the end of the file. To fix, I just reverted the change to that particular line. Also add a test that tokens have locations that are in order
…on-data Fix incorrect location data in OUTDENT nodes
In f609036, a line was changed from
if length > 0 then (length - 1) else 0toMath.max 0, length - 1. However,in some cases, the
lengthvariable can beundefined. The previous code wouldcorrectly compute
lastCharacteras 0, but the new code would compute it asNaN. This would cause trouble later on: the end location would just be the endof the current chunk, which would be incorrect. This confuses tools that rely on
the CoffeeScript parser and need the location info to be correct.
Here's a specific case where the parser was behaving incorrectly:
The OUTDENT tokens after the
fhad an undefined length, so theNaNmade itso the end location was at the end of the file. That meant that various nodes in
the AST, like the
returnnode, would incorrectly have an end location at theend of the file.
To fix, I just reverted the change to that particular line.