bpo-30302 Make timedelta.__repr__ more informative.

[musically-ut]

Currently, the default implementation of datetime.datetime.repr (the default output string produced at the console/IPython) gives a rather cryptic output:

from datetime import datetime as D
D.fromtimestamp(1390953543.1) - D.fromtimestamp(1121871596)
# datetime.timedelta(3114, 28747, 100000)

For the uninitiated, it is not obvious that the numeric values here are days, seconds and microsecond respectively.

Would there be any pushback against changing this to:

# datetime.timedelta(days=3114, seconds=28747, microseconds=100000)

?

https://bugs.python.org/issue30302

[the-knights-who-say-ni]

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA).

Unfortunately our records indicate you have not signed the CLA. For legal reasons we need you to sign this before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

Thanks again to your contribution and we look forward to looking at it!

[mention-bot]

@musically-ut, thanks for your PR! By analyzing the history of the files in this pull request, we identified @tim-one, @abalkin and @serhiy-storchaka to be potential reviewers.

[vadmium]

As well as tests, the documentation needs reviewing. It actually specifies the repr return value.

[musically-ut]

@vadmium Good point. I'm assuming that the documentation is not limited to only the doc strings (actually, I couldn't find an instance there on a cursory look).

Is there an easy way to check that I have caught all cases in the extended documentation?

[musically-ut]

@vadmium I see what you meant by:

It actually specifies the repr return value.

here: https://github.com/python/cpython/blob/3.6/Doc/library/datetime.rst

So we'll have to consider the risk of someone's code breaking because it depends on parsing the repr representation of timedelta?

---

Update: Related mail thread addresses the question somewhat: https://marc.info/?l=python-dev&m=145065097122111&w=2

[vstinner]

Concatenate strings using += is inefficient. Moreover, you have to handle comma manually. Please rewrite this code using a list (args = [] ... args.append(...)), and then join it using: ', '.join(args).

[musically-ut]

Thanks! Will do that.

[vstinner]

Why are these tests commented? Either enable them or remove them.

[musically-ut]

Martin wanted the handling of negative attributes to change while I am of the opinion that they shouldn't. So we may end up discussing these tests later.

However, I'll remove them from the PR presently.

[vstinner]

Hum, the >= ""< part is really strange (remove it?).

Moreover, why 64 bytes? Microseconds cannot be larger than 999 999, so 6 digits + 1 null byte, no?

[musically-ut]

I got the 64 bytes number from here, which could also be 6 digits + 1. However, in light of your next comment, we may not end up needing this at all.

[vstinner]

Hum, no module nor qualified name here?

[musically-ut]

tp_name contains the fully qualified name.

[vstinner]

Hum, the code may be simpler if you use a PyList and ", ".join(), as proposed for the Python code. You can use PyUnicode_FromString("seconds=%d", GET_TD_SECONDS(self)) to build an argument. It would avoid to have to compute a buffer size (and taking the risk of a buffer overflow).

[musically-ut]

I've implemented a version using PyList and PyUnicode. I don't think I'm leaking any memory there, but since this is the first time I'm touching PyObjects, I would really appreciate a second opinion on it.

[serhiy-storchaka]

The result of C API function calls always should be checked for error.

There are memory leaks.

[serhiy-storchaka]

args_string is leaked.

[serhiy-storchaka]

The result of PyUnicode_FromString(", ") is leaked.

[serhiy-storchaka]

PyUnicode_FromFormat() and PyList_Append() can fail.

The result of PyUnicode_FromFormat() is leaked.

[serhiy-storchaka]

PyList_New(0) can fail.

[serhiy-storchaka]

Just inline arg_str. This is simple expression.

[musically-ut]

Done. Thanks!

[serhiy-storchaka]

Why this is changed?

[musically-ut]

Because the tests were only running for the C implementation, not for the Python
implementation. This was because test_classes was being overwritten inside the for loop. With this change, the tests run twice: once for the pure Python module and once for the C extension.

[vstinner]

Please fix the loop rather than renaming this variable.

[vstinner]

Ping?

[musically-ut]

Ah, sorry, I didn't recognize that it was a comment on this review, Github didn't allow me to reply to this from where I actually saw it.

So, yes, the loop is correct, the only thing it was doing incorrectly was overwriting the test_classes variable when it moved from '_Pure' classes to '_Fast' classes. It does need to name them differently (I'm not sure why the names of the classes don't actually change in the verbose output, that seems to take me down the test_suite rabbit hole to figure out whether __name__ is the correct attribute to change or not). However, the behavior of the tests is correct now, i.e. it tests both implementations in sequence.

I'm investigating whether there is a simple fix to make the correct version of classes to show up in the tests.

[musically-ut]

Done.

[serhiy-storchaka]

Because the tests were only running for the C implementation, not for the Python
implementation.

This looks like a bug. Please open a new issue for it. This should be fixed in all versions.

[musically-ut]

Good point. I'll check which versions are affected (i.e. the branches on the cpython repository, right?) and open another bug. What should I do with this fix, though?

If I let it remain here, it may potentially lead to a merge conflict?

[musically-ut]

I've created http://bugs.python.org/issue30822

[serhiy-storchaka]

After merging the PR that fixes the tests this PR should be merged with master. Merge conflicts that are not resolved automatically should be resolved manually.

[serhiy-storchaka]

You can decref days here. This will simplify the cases for seconds and microseconds.

[serhiy-storchaka]

Just Py_DECREF. days is not NULL.

Or you can merge two ifs:

if (days == NULL || PyList_Append(args, days) < 0) {
    Py_DECREF(args);
    Py_XDECREF(days);
    return NULL;
}

[serhiy-storchaka]

You can decref sep and args just here.

[musically-ut]

@serhiy-storchaka Thanks for the suggestions! The code looks much cleaner and more concise now. Have a look!

[vstinner]

Prefer don't remove documentation, but update it. If you don't want to describe the format, just say something like "Returns the representation of the time delta as a string."

[musically-ut]

Sounds reasonable. Done.

[vstinner]

I don't think that it's worth it to add so many test cases. Just keep 3 examples, 1 negative value, the timedelta(0), and you're done no?

[musically-ut]

Some of the tests were added to differentiate the current implementation from the old one: i.e. no leading days=0 or seconds=0 and to highlight the current handling of negative values (i.e. the normalization of the attributes, as described in the docs). I'll remove the ones which don't test anything unique not covered elsewhere.

[vstinner]

Please fix the loop rather than renaming this variable.

[vstinner]

Please move "Py_DECREF(args); return NULL;" to a new "error:" label at the end, and use "goto error;" here, to factorize the code (duplicated 3 times).

[musically-ut]

Done.

[vstinner]

Replace this with:

if not args:
args.append('0')

to prevent code duplication.

[musically-ut]

👍

[vstinner]

"if not args:" is better

[vstinner]

Ditto.

[musically-ut]

👍

[vstinner]

Please replace PyList_Size() with PyList_GET_SIZE().

PyList_Size() can fail, PyList_GET_SIZE() cannot.

[vstinner]

I don't recall if you reply to my question: does this include the module name? Does it use the qualified name?

[musically-ut]

Yes, it uses the fully qualified name. Otherwise, the tests would have caught it. I did reply to it, but Github probably collapsed the comment.

[vstinner]

Ok, cool :-)

[vstinner]

This line seems weird. Did you try to compile the doc? make -C Doc html

Why did you touch this doc?

[musically-ut]

Ah, crap. This was an illusion created by Vim's soft-wrapping. git add -p too broke the lines weirdly so I wasn't able to catch it. Am running make html now to verify the output. Sorry about missing this.

[vstinner]

"if not args:" is better

[vstinner]

IMHO the empty lines doesn't help readability, you can remove them in all the code filling args.

[musically-ut]

👍

[vstinner]

Ping?

[vstinner]

Please replace PyList_Size() with PyList_GET_SIZE().

PyList_Size() can fail, PyList_GET_SIZE() cannot.

[vstinner]

remove this useless empty line

[musically-ut]

👍

[vstinner]

Ok, cool :-)

[vstinner]

You might remove this empty line for readability, collapse the logical block.

[musically-ut]

👍

[vstinner]

LGTM, but I would like to see a review of @serhiy-storchaka and/or @vadmium since we had some disagreement on the repr() output.

[serhiy-storchaka]

The C code looks correct to me, but a little cumbersome. See my comment on the tracker.

Please add your name in Misc/ACKS if it still is not added and an entry in Misc/NEWS.d/.

[serhiy-storchaka]

Because the tests were only running for the C implementation, not for the Python
implementation.

This looks like a bug. Please open a new issue for it. This should be fixed in all versions.

[serhiy-storchaka]

It is not clear how this representation differs from the above one. May be add "in the calling form" or something like?

[musically-ut]

Returns the representation of the :class:timedelta object such that t == eval(repr(t)).

?

[vstinner]

Still LGTM with the new C code ;-)

[serhiy-storchaka]

Yet few nitpicks.

[serhiy-storchaka]

After merging the PR that fixes the tests this PR should be merged with master. Merge conflicts that are not resolved automatically should be resolved manually.

[serhiy-storchaka]

const char*

[serhiy-storchaka]

I think all formatted arguments may be written in one line. This is more compact.

[serhiy-storchaka]

Names are ordered in alphabetical order. Your name should be under the U letter.

[musically-ut]

t == eval(repr(t)) isn't necessarily true (e.g. when datetime is aliased during import). So this part of the documentation would need to change again.

[musically-ut]

I've rebased the PR on the current master.