`std.Progress`: possible simplifications and ideas

After discovering std.Progress today I noticed that this is how it's resetting the terminal cursor
to the column where the cursor was before the line was written and how it's clearing that line: https://github.com/ziglang/zig/blob/master/lib/std/Progress.zig#L206-L254

• All this could be greatly simplified by printing '\r' which in terminals always resets the cursor to column 0. This works on Windows too. So when you print "hello world\rworld", it will print as world world!
  As for the clearing of the previous content, let's say we have hello world on the terminal but now we want to show only hello on the same line. We would simply print "\rhello" + 6 more spaces to erase world too.
  All this would greatly simplify the code and avoid escape sequences altogether. It'd be much more portable and reliable. It would also no longer require any special casing for Windows. Performance and printing speed might improve too.

Now I would really like to start working on this but the blocker I have is that doing it this way will no longer allow printing multiple progresses on the same row or else it'll look messed up. This would no longer be possible:

Test [1/1] test "basic functionality"... [3/100] climbing towers [3/5]

so it'd have to be like this:

Test [1/1] test "basic functionality"...
[3/100] climbing towers [3/5]

So would it be fine to allow only one progress line per row?

I'd specifically be interested to hear an opinion from the original designer of std.Progress, @andrewrk.

Other things that could be done include

• Adding an option to the compiler to disable the progress bar/any output during compilation (does this already exist?) because I can definitely see Zig being useful as a language for other languages to compile to (as opposed to compiling to C) so if they want to do that by invoking zig then it'd be nice to disable that output.
• Removing the space after the ellipsis (this would make the most sense if we go with my idea above of allowing only one progress per row in order to simplify the code using '\r')
• Adding an actual loading bar-looking thing like this?

  [=====     ] [50/100] reticulating splines [5/5]
  

  There are a lot of flashy CLI libs etc. out there that make use of Unicode characters and stuff to make the their loading bars look really cool (think box-drawing characters etc.) but I think we should limit ourselves to ASCII characters here to make sure it works in as many environments as possible because not all Unicode characters are supported everywhere. Also, I don't think doing this provides any actual value so I think these fancy loading bars are probably best left to some external library and we should keep the current progress bar design.

[andrewrk]

Are you willing to experiment with this and then show some demos of the difference? I think if we look some demos it will be fairly easy to assess and agree on what changes are nice.

From my perspective, the main problem with std.Progress right now is that if the line goes beyond the end of the terminal, it causes a newline, and then the mechanism for resetting the cursor doesn't work, causing garbage to be emitted. Does the \r strategy solve this problem?

From my perspective, the main problem with std.Progress right now is that if the line goes beyond the end of the terminal, it causes a newline, and then the mechanism for resetting the cursor doesn't work, causing garbage to be emitted. Does the \r strategy solve this problem?

I see.
Hmm, no, it doesn't look like \r would fix this problem either.

const std = @import("std");

pub fn main() void {
    std.debug.print("hello world this is a very loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong string\n",.{});
    std.debug.print("hello world this is a very loooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooong string\rX\n",.{});
}

In a sufficiently sized terminal the second line's printed output would start with Xello but in a terminal that is too small, it doesn't:

I think this is a pretty difficult problem that might be solvable by actually detecting if the terminal window was resized (e.g. by detecting the SIGWINCH signal on POSIX systems etc.) but if we go that way then the implementation will become very complicated. I don't think we want that at all.

Unless I'm missing some actually easy way to fix this I think we might want to start accepting that we can't fix this and instead should look to make progress lines smaller.
One way my proposal could help is that if we only allow one progress line per terminal row and remove the trailing space then it should be less common for the progress line to cross the terminal width in the first place. Additionally, we could replace the 3-character ellipsis ... with the Unicode character …. But this would cause another ASCII compatibility problem so we probably don't want to do that.

I think I will move on then with my plans for now and see what I can come up with.

Hmm everything seemed to have worked fine for now until it came to collaboration between multiple programs. The test in Progress.zig "passes" (prints fine) as long as I run them myself and not with zig test which itself also uses std.Progress because then it clashes with my test because zig test's std.Progress has its own buffer and my test's std.Progress instance has its own buffer, and so they don't work together and \r always going to column 0 causes a problem.
Well, it looks it might not be possible to use \r here.

I mean we could of course use newlines at the end of the progress lines which would solve that problem but then Zig compiler, test etc. output will become very long and I don't think that's what we want.

[marler8997]

A couple resources that would be helpful here are the terminfo/termcap man pages to see all the options for many posixy terminals that exist in the world:

• https://man7.org/linux/man-pages/man5/terminfo.5.html
• https://man7.org/linux/man-pages/man5/termcap.5.html

These manuals describe 2 competing standards for specifying terminal capabilities/limitations. Libraries like "ncurses" will identify the current terminal, then grab the "termininfo" or "termcap" about the terminal and use it to customize the implementation for high-level operations such as "moving the cursor to the start of the line".

Actually, @topolarity had the great idea of using an escape sequence to get the current cursor X position, which will allow us to go with the \r approach. It seems to be widely supported: https://terminalguide.namepad.de/seq/csi_sn-6/
Additionally, we thought about using ioctl to fetch the terminal width once at the start to use for the output buffer.

@marler8997 thanks for the resources!

Alright I'm stopping my attempts at this for now.
Here are my branches if anyone wants to use them as a reference or whatever.
https://github.com/r00ster91/zig/commits/niceprogress
https://github.com/r00ster91/zig/commits/betterprogress

[daurnimator]

See also #2931 and the closed PR #3872