Common TestSuite for MIP backends

[mkoeppe]

The MIP backends should be tested using a common TestSuite. Right now each backend uses its own doctests, which have slightly diverged from each other, so there is nothing (other than the fact that they were the result of copy-paste from each other) that ensures consistency. (#20296 fixes several wrong doctests in GenericBackend, from which I tried to copy-paste, for example.)

Depends on #20326

CC: @dimpase @nathanncohen @videlec @vbraun @nthiery

Component: numerical

Author: Matthias Koeppe

Branch/Commit: 7138fa0

Reviewer: Thierry Monteil, Dima Pasechnik

Issue created by migration from https://trac.sagemath.org/ticket/20323

[dimpase]

comment:1

within the existing framework it's easy to create a separate module, say, lp_backend_tests, and do such tests there.
Not sure how to minimize the bloat, though, as each function should have doctests...

Note that total consistency is hard to get, as each backend has its own ideas on how, say, to number and order constraints. E.g., IIRC, gurobi does automatic conversions of some particular kinds of constraints.

[mkoeppe]

Branch: u/mkoeppe/common_testsuite_for_mip_backends

[mkoeppe]

Commit: f5f69bd

[mkoeppe]

comment:3

Not ready for review yet, but I'm hoping that perhaps someone familiar with the TestSuite mechanism could tell me whether I'm on the right track with this initial patch on this ticket.

---

New commits:

f5f69bd

Use TestSuite for testing MIP backends

[tscrim]

comment:4

It looks like you are. Any method that starts with _test gets run by TestSuite. Although I am not really in favor of any test mutating the object in question.

Also, is there a reason why GenericBackend needs to inherit from SageObject (as opposed to just the Python object)? I am pretty sure this is strictly needed...

[mkoeppe]

comment:5

Thanks for taking a look, Travis.

Replying to @tscrim:

Also, is there a reason why GenericBackend needs to inherit from SageObject (as opposed to just the Python object)?

SageObject provides some of the basic testing infrastructure, such as ._test_not_implemented_methods.
Well, there's also sage.misc.sage_unittest.PythonObjectWithTests, but I can't seem to cimport it to make it a base class of GenericBackend. So I guess I'll stick with SageObject.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

d2a8a26	Add comment
5034459	More _test_... functions

[sagetrac-git]

Changed commit from f5f69bd to 5034459

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

2f8d672	InteractiveLPBackend: Remove commented-out code
3a26453	InteractiveLPBackend: Expand example with algebraic numbers
b7f1ed8	get_solver: Add another doctest
8c1fd4a	get_solver: Fix last doctest
d23c57f	Merge branch 't/20296/mixedintegerlinearprogram__new_backend_using_interactivelpproblem' into lp_fixes
3f827ea	Use TestSuite for testing MIP backends
ec68338	Add comment
1d617b7	More _test_... functions
3c6a759	get_solver: Make doctest work no matter whether backends are Python or Sage objects
06f1482	Run testsuite with all MIP backends

[sagetrac-git]

Changed commit from 5034459 to 06f1482

[mkoeppe]

comment:8

Branch is on top of various closed/reviewed LP tickets from #20302. Will rebase on top of next 'develop' later.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,2 @@
- I think the backends should be tested using a common `TestSuite`. Right now each backend uses its own doctests, which have slightly diverged from each other, so there is nothing (other than the fact that they were the result of copy-paste from each other) that ensures consistency. (#20296 fixes several wrong doctests in `GenericBackend`, from which I tried to copy-paste, for example.)
+The MIP backends should be tested using a common `TestSuite`. Right now each backend uses its own doctests, which have slightly diverged from each other, so there is nothing (other than the fact that they were the result of copy-paste from each other) that ensures consistency. (#20296 fixes several wrong doctests in `GenericBackend`, from which I tried to copy-paste, for example.)
 
-Pointers on how to do this would be welcome.

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

af4bcd5

GenericBackend._test_add_variables(): Add tests from CVXOPTBackend

[sagetrac-git]

Changed commit from 06f1482 to af4bcd5

[nthiery]

comment:11

Hi Matthias!

Just had a brief look at your TestSuite usage. This sounds very sensible indeed.

Inheriting from PythonObjectWithTests would indeed make sense; however since it's currently a plain Python class, one can't inherit from it in a Cython class. This could be fixed though.

Cheers,
Nicolas