Simplify TestSuite's some_elements role and logic (no random sampling)

[nthiery]

Since #14284, TestSuite (more precisely InstanceTester.some_elements) tries to be fancy by choosing "some elements" using a random sample. The random sample is built using Python's random.sample, which requires its input to be a Sequence (i.e. the i-th element can be fetched with o[i]), or some dict-like object. This can get brittle with inputs where __getitem__ is used for other purposes, or where unranking is just computationally expensive. The some_elements method also assumes __len__ to be implemented and cheap enough.

Example:

sage: FF = IntegerModRing(29)  # needs to be >21 otherwise random.sample uses a different strategy
sage: tester = FF._tester(elements=FF, max_runs=5)
sage: list(tester.some_elements())
...
ValueError: first letter of variable name must be a letter

This ticket reduces the role of InstanceTester.some_elements to just do some argument parsing and ensure that at most "max_run" elements are returned. It only requires the input to be iterable.

If we want to have fancy random samples, we should define a specific protocol (typically P.sample()) for it, or just let parents decide on the strategy by defining some_elements appropriately.

This was originaly analyzed in #15919.

TODO: decide whether InstanceTester.some_elements should return a list or an iterator. A list would be more user friendly (though that's not critical since this is pretty low level) and maybe less error-prone (if a _test_method attempt to reuse the result twice). On the other hand, in case of failure of a _test_method, using an iterator would waste a bit less time before the failure occurs (no need to build all the elements).

CC: @mezzarobba @sagetrac-sage-combinat @roed314 @saraedum

Component: doctest framework

Author: Nicolas M. Thiéry

Branch: 2486f80

Reviewer: Travis Scrimshaw

Issue created by migration from https://trac.sagemath.org/ticket/16244

[nthiery]

Branch: u/nthiery/simplify_testsuite_s_some_elements_role_and_logic__no_random_sampling_

[nthiery]

Commit: 2486f80

[nthiery]

New commits:

2486f80

#16244: Simplify the role and logic of TestSuite's some_elements (no random sampling)

[nthiery]

Description changed:

--- 
+++ 
@@ -1,19 +1,15 @@
-Currently `TestSuite` (more precisely `InstanceTester.some_elements`) tries to be fancy by choosing "some elements" using a random sample. The random sample is built using Python's `random.sample`, which requires its input to be a Sequence (the i-th element can be fetched with o[i]), or some dict-like object. This can get brittle with other kinds of inputs where `__getitem__` is used for other purposes, or where unranking is just computationally expensive. The `some_elements` method also assumes `__len__` to be implemented and cheap enough.
+Since #14284, `TestSuite` (more precisely `InstanceTester.some_elements`) tries to be fancy by choosing "some elements" using a random sample. The random sample is built using Python's `random.sample`, which requires its input to be a Sequence (i.e. the i-th element can be fetched with o[i]), or some dict-like object. This can get brittle with inputs where `__getitem__` is used for other purposes, or where unranking is just computationally expensive. The `some_elements` method also assumes `__len__` to be implemented and cheap enough.
 
 Example:
 
 ```
-FF = IntegerModRing(29)
-tester = FF._tester()
-list(tester.some_elements(CartesianProduct(FF, FF, FF)))
-*bang*
+sage: FF = IntegerModRing(29)  # needs to be >21 otherwise random.sample uses a different strategy
+sage: tester = FF._tester(elements=FF, max_runs=5)
+sage: list(tester.some_elements())
+...
+ValueError: first letter of variable name must be a letter
 ```
 
-In fact the following is close to fail, and just works because the error raised during the sample is caught.
-
-```
-tester.some_elements(FF)
-```
 
 This ticket reduces the role of `InstanceTester.some_elements` to just do some argument parsing and ensure that at most "max_run" elements are returned. It only requires the input to be iterable.
 

[nthiery]

Description changed:

--- 
+++ 
@@ -13,6 +13,8 @@
 
 This ticket reduces the role of `InstanceTester.some_elements` to just do some argument parsing and ensure that at most "max_run" elements are returned. It only requires the input to be iterable.
 
-If we want to have fancy random samples, we should define a specific protocol (typically P.sample()) for it, or just let parents decide on the appropriate strategy by defining some_elements appropriately.
+If we want to have fancy random samples, we should define a specific protocol (typically P.sample()) for it, or just let parents decide on the strategy by defining some_elements appropriately.
 
 This was originaly analyzed in #15919.
+
+TODO: decide whether InstanceTester.some_elements should return a list or an iterator. A list would be more user friendly (though that's not critical since this is pretty low level) and maybe less error-prone (if a _test_method attempt to reuse the result twice). On the other hand, in case of failure of a _test_method, using an iterator would waste a bit less time before the failure occurs (no need to build all the elements). 

[tscrim]

Reviewer: Travis Scrimshaw

[tscrim]

comment:5

LGTM (also with #15919).