Request for Ultest equivalent to vim-test's :TestSuite

[DanHatesNumbers]

Thanks for writing this plugin! Similar to #69 I'm wondering about the possibility of adding support for running an entire test suite like vim-test's :TestSuite.

If this would be a reasonable first issue, I'd be happy to try and add support and raise a PR if you don't mind giving me an idea of where to start.

[rcarriga]

I've actually thought about this a good bit. It's a fairly large amount of work and would require some larger changes than I'd be comfortable having someone else do entirely (appreciate the offer though! 😄). Since you've opened this issue I might as well document my thoughts here...

In my head ultest would ideally mirror something like VSCode where it can find all the test files in your workspace, along with all the tests inside.

As part of that there's a few changes needed:

1. Change summary window to act more like a file tree. It needs to group files together under common parent directories so that the user can expand/collapse what they like. Ideally I think custom folding should be added as well, using vim's built-in folding isn't the prettiest and now would be required much more.
2. Being able to parse test results as a suite. This is the largest change as it needs to be done per runner. There is no way ultest can run suites as separate tests as it does for test files (if it doesn't support parsing output of the runner), it would destroy performance. For something like pytest this is quite easy but others like go-test are quite a bit tougher because it doesn't output the file names.
3. Find the test directory in the current workspace and parse all the files in it to first see if it's a test file (which needs to be done synchronously because it's using vim-test code 😞) and then read all tests (fine because it's async). I think this would require a setting ultest_test_directories which we could default to something like ["tests", "test", "spec"].

I definitely want to do 2 myself and most likely 3. Changing the summary window is an isolated thing though and I'm open to PRs for it. I'm pretty busy at the moment so I don't have much time to work on personal projects so I'll take the help! Leaving this open to anyone really, happy to discuss as well to get things started.

[joshpetit]

Wow I love the work thats going into this! Thanks a lot @rcarriga!