Model lexer is still wrong

[emberian]

Tracking issue for all known problems with the lexer.

Todo:

• antlr's lack of unicode is disturbing #15679 antlr's lack of unicode is disturbing
• Model lexer doesn't handle range correctly #15877 Model lexer doesn't handle range correctly
• Weirdness with shebang between model and rustc #15878 Weirdness with shebang between model and rustc
• Model lexer doesn't handle question marks #15879 Model lexer doesn't handle question marks
• Model lexer doesn't handle ... something? #15881 Model lexer doesn't handle ... something?
• Model lexer doesn't handle CRLF correctly #15882 Model lexer doesn't handle CRLF correctly
• Remove any remaining ignore-lexer-tests.

[fhahn]

Does it stilll make sense to keep the antlr lexer after the flex/bison parser was added in #21452 ?

[emberian]

I initially wrote this lexer to serve as the reference because it is far simpler than the flex one.