List of tests to add

[huguesdevimeux]

At the moment we only have last-frames tests. Although they are useful, it's not enough. We would need to have tests for cli flags, that check videos, etc. This issue will keep a list of tests that need to be added. Please make more suggestions for tests!

Tests that need to be added

• We should have at least one test for each CLI flag.
• We should test that the cascading priorities in the config system work as expected. Ties in with Add a manim cfg-show sub command #201
• We should create a test to certify a clean install. See Create a test to certify a clean install #11.
• Testing video content
• Test that manim runs correctly on all environments/prompts/versions Fixed bug that broke some Windows CLIs #336
• Regression test for remove_stray_braces quickfix #240
• Test z-order stuff Bug in handling of z_index #327
• Test to ensure MovingCamera and ZoomedScene let the camera move as they should. From Fix MovingCamera and ZoomedScene #388 .
• Test the functionality of asking the user for which scenes to render add missing import #523
• There are no tests for the -s flag or the -t flag
• Test for add_fixed_in_frame_mobjects which is used in 3D scenes Fixing add_fixed_in_frame_mobjects #569, currently broken
• Tests for SVG images
• @XorUnison Once video tests are implemented reverse_direction should get one
• @XorUnison Test for Brace
• @PhilippImhof Graphical unit test for Text and MarkupText using a fixed font (TTF)

[naveen521kk]

I recently came up with some good things like coverage.py which is great. What it actually does is checks how many parts of code is executed when the tests are running and how many doesn't(for example checks how many if condition code are not executed as so on). It make beautiful html report or we can something like https://codecov.io for managing that. Pretty much, it will help us determine how good our tests are and to prevent code breakage. I suggest we should set it up soon and it isn't hard to it.

[leotrs]

Before looking into code coverage tools, I would like to see more tests.

[csrabak]

@leotrs :

The two subjects are well interconnected, we could say they're conjoined twins ‼️

Adding tests if do not exercise all the control flow paths may give a sense of security when in fact are only redundancies.

That's why coverage should be considered as integral part of the testing exercise.

my .01999....

[huguesdevimeux]

That's why coverage should be considered as integral part of the testing exercise.

I agree with that

[naveen521kk]

Probably we can open a new issue for that. @huguesdevimeux

[leotrs]

I agree that coverage should be taken into account, I just meant to say that right now the priority should go to writing tests. If anyone is looking into implementing coverage, that time would 100% be better spent by implementing more tests, as of right now.

Having said that, if anyone wants to do that now, then by all means. But deciding upon one more tool should be discussed in a separate issue, and we should talk about a tool that works for doctests as well, at least.