Parser: improve error message handling

[chqrlie]

• use single on_error handler with error level and message arguments
• remove Warning token type, never returned anyway.
• improve #error and #warning message parsing consistency
• make num_error messages non fatal
• fix #warning behavior

[bvdberg]

It seems to make parsing slower... What was it that you were trying to improve?

[chqrlie]

It seems to make parsing slower...

This is surprising, the changes should not affect the inner loops, only the error cases.

What was it that you were trying to improve?

I was just trying to improve the consistency:

• both warnings and errors should be produced synchronously,
• #warning used to output an error with on_warning, itself using diag.error instead of diag.warn
• num_error are errors, not warnings, but non fatal.
• #error returned an error token without showing the error message.

[bvdberg]

debug leftover?

[chqrlie]

More anticipation of future feature :)

String buffers currently null terminate the array for each individual call. I suggest only adding the null terminator when the pointer or the size is actually needed. This would reduce the number of redundant writes and potentially improve the inlining of single character additions.

[bvdberg]

string_buffer.create_static does a malloc (surprised me as well), but old code seemed to be much simpler. If that used the t.error_msg buffer instead of a stack... The string buffer is now barely used (it does a memcopy internally). Also why parse to Eof instead of newline?

[chqrlie]

string_buffer.create_static does a malloc (surprised me as well),

The string_buffer.Buf object would not need to be allocated in this case, as well as many other ones, if we had separate string_buffer.Buf.init(Buf* b, const char *buf, u32 size, bool reallocate) function called by both string_buffer.create and string_buffer.create_static.

but old code seemed to be much simpler.

The #error and #warning features should use preprocessed tokens like #if for multiple reasons:

• for consistency with the C parser.
• to simplify language support in editors and code colorizers (simpler if we follow the already supported C semantics)
• so comments are parsed as such on these lines and thus can be used by the tester, or the programmer.

If that used the t.error_msg buffer instead of a stack... The string buffer is now barely used (it does a memcpy internally).

It does a bunch of strcpy and strcat with truncation. We could indeed write and use strlcpy-like functions instead.

Also why parse toEof instead of newline?

Because t.lex_preproc(result) returns Eof on both the actual end of file and the first newline encountered. lex_preproc is a wrapper for lex_internal that sets the stop_at_eol for this very purpose. I shall add a comment to clarify this side-effect.