Regex look-ahead expressions do not work properly (begin)

[jandamm]

This issue is somewhat related to #1550 where I commented too.
But I decided to give a better/clearer example how to reproduce the issue.

The problem is that grouped regexes are combined with into one regex using | (OR). After the match, it is not checked if the match really matches to the current group.

To reproduce this issue, you can use this parser and replace the swift.js one with it.

module.exports = function(hljs) {
  const VAR1 = { 
    className: 'variant_1',
    begin: /\b[a-z][A-Za-z0-9]*\b(?!\()/g
  };
  const VAR2 = {
    className: 'variant_2',
    begin: /\b[a-z]/g
  };
  return { contains: [{
    begin: /[A-Za-z0-9]\./, end: /[\.|\,|\)|\n]/,
    contains: [VAR1, VAR2]
}]};}

Now you match this code:

<pre><code class="language-swift">
  a.make)
  a.bake()
</code></pre>

And it will turn into this:

<pre><code class="language-swift">
  a.<span class="hljs-variant_1">make</span>)
  a.<span class="hljs-variant_1">b</span>ake()
</code></pre>

But actually m matches variant_2.

If you remove VAR2 from the contains array it works fine. (make)
Same goes if you remove VAR1. (m, b)

[joshgoebel]

<pre><code class="language-swift">
  a.make)
  a.bake()
</code></pre>

Can you please clarify what you believe the correct output should be for this given the above rules and why? I believe it just tries the contains patterns in order looking for a match.

The first occurrence matches variant_1, so it stops looking further... the second does not match variant_1 so it instead matches variant_2.

Also interested in how this relates to #1550.

[jandamm]

Over a year later I'm not a 100% sure but it seems to me that it should be

  a.<span class="hljs-variant_2">b</span>ake()

instead of

  a.<span class="hljs-variant_1">b</span>ake()

[joshgoebel]

Is that because you think contains is a SEQUENCED list of matches that are REQUIRED?

[jandamm]

That is because I think that I can give a REGEX with a className and if it is matched the className is applied instead of the className that does not match.

[joshgoebel]

Ah I see now. You're right.

The problem is /\b[a-z][A-Za-z0-9]*\b(?!\()/g, in that the way the parser is written the negative look ahead doesn't have a chance to do its thing... so it will always pass - so variant 1 is chosen. I'm not 100% sure why it isn't being greedier though. (though I think I can guess)

The parser takes ALL the matches for a given mode and the (terminators) of those and then scans thru the text from one terminator to the next... The problem isn't lookaheads don't generate any "output" so when the parser goes to analyze the match it only "sees":

• "make"
• "b"

And both those match the first rule.

[joshgoebel]

See my comment on #1349. I suggested a way we could fix this. (I'm talking about positive look aheads, but it's all the same I think).

[joshgoebel]

If you test this against master you should find it now works.