Lexer: `a = ; 5`: Suppressed newline should be unsuppressed by semicolon

[vendethiel]

(from #3053)

# rule : 
'Assignable = TERMINATOR Expression'
# supposedly allowing that :
a =
5
# but also allows this
a = ; 5

[marchaefner]

That one is fun. (This is a function: f = (a; b; c) ->)

We could remove TERMINATOR tokens (if they're not semicolons) after = and COMPOUND_ASSIGN in the rewriter (and save two rules in the parser). For commas (as in the arglist above) it's a bit more complicated though.

[jashkenas]

Right, the other 'correct' way to fix this would be to treat = like other 'unfinished' bits, and have it suppress the newline, but I bet that'll cause more problems than it fixes.

[vendethiel]

This is a function: f = (a; b; c) ->

I guess it's to allow a = (x; y = f z) -> x

[marchaefner]

@Nami-Doc: Ha, very nice. It's actually useful. (We don't need no stinking parens! ;)

I think we should keep those OptComma TERMINATOR cases, they seem way too hard to fix anyway. (I really doubt this behavior is intended.)

@jashkenas: Better not. The (premature) "unfinished" bits don't not play nice with the rewriter (e.g. #3199).

[erisdev]

Haha, yeah no, suppressing newlines after = would break implicit object literals, wouldn't it?

[marchaefner]

@erisdiscord: Yes, but only in single-liners (IIRC), e.g.

  if c then x =
    a:b
    c:d

[erisdev]

@marchaefner would it not also break them in this case? or would the indentation be sufficient?

someNumbers =
  arbitraryNumber: 37
  randomNumber: Math.random()

[marchaefner]

@erisdiscord: The unfinished logic (plus the corresponding indebt "magic") also removes the indentation. normalizeLines (where single-liners are handled) subsequently adds indentation until it encounters a TERMINATOR, effectively splitting the object. (Plug: coffee-tap let's you look at the the un-rewritten token stream.)

[jashkenas]

Plug: coffee-tap let's you look at the the un-rewritten token stream.)

As does the helpful comment at the top of the rewriter file. You can drop that sucker in at any stage of the rewriter's passes, to see how well things are moving along.

[GeoffreyBooth]

I’m assuming the solution for this is to have the semicolon “un-suppress” the newline that gets suppressed by the open =.