consider switching to `compass run` instead of `./run.py`

[xylar]

@matthewhoffman came up with a great suggestion of no longer generating the ./run.py scripts (which can't usefully be edited to make changes to the test case or step anyway) and instead going with a command compass run that runs a suite, test case or step.

The main consideration is how the information unique to the suite, test case or step that is currently in run.py gets stored and retrieved. Presumably, the pickle files for test case and steps could be given more generic names and then compass could recognize whether a directory is a test case or a step by whether there's a test_case.pickle or step.pickle.

Test suites are a little more complicated because several suites can get set up in the same directory. In that case, it probably makes sense to have a pickle file for each suite and call compass run <suite>, where <suite> is the (hopefully unique) name of the suite.

[matthewhoffman]

How are you imagining compass run being invoked? Analogously to compass setup or compass suite? Would you have to run it from the workdir such that it is implied that what you want to run is the dir you are currently in (be it suite, testgroup, testcase, or step)?

Maybe it would be more explicit to run compass run directly on the pickle file? Then it's clear what you are trying to run.

[xylar]

This has been implemented and tested here:
https://github.com/xylar/compass/tree/switch_to_compass_run

94 additions and 155 deletions.

That's always a plus!

[xylar]

@matthewhoffman,

How are you imagining compass run being invoked? Analogously to compass setup or compass suite? Would you have to run it from the workdir such that it is implied that what you want to run is the dir you are currently in (be it suite, testgroup, testcase, or step)?

Yes, you run in the work directory. Currently, there's no way to run all the test cases in a test group, though that's an interesting idea! But for test cases and steps, it just recognizes the test_case.pickle or step.pickle and runs the test case or step. You just run:

python -m compass run

without any further arguments.

Since multiple suites can be run from the same directory, you have to be in the root of the work directory and then give the name of a suite to run:

python -m compass run nightly

Maybe it would be more explicit to run compass run directly on the pickle file? Then it's clear what you are trying to run.

Yeah, that would be an option but I sure like not having to worry about that with the current ./run.py approach. Given that it's really easy for compass to figure out what test case or step it's meant to run as long as the name of the pickle file is generic, that's an easier and more convenient way to go.

[xylar]

This work has the added benefit of letting the suite, testcase and step packages become modules instead because the template files aren't needed anymore.

[matthewhoffman]

@xylar , wow that was surprisingly quick to implement! The way you've implemented it makes sense to me, and I agree it is cleaner to not have to reference the pickle files (which are meant to be ignored by users) - you convinced me that there is not ambiguity about what compass should be running when invoked from various locations. I also like the reduction in complexity that comes with fewer lines of code, eliminating the jinja templates, eliminating auto-generated scripts, and reducing a few packages to modules. I'll keep these proposed changes in mind as I finish reviewing the v1 PR.