Breaking change: DateTimeField validation parsing

[RignonNoel]

Checklist

• Raised initially as discussion Breaking change: DateTimeField validation parsing #9156
• This cannot be dealt with as a third party library. (We prefer new functionality to be in the form of third party libraries where possible.)
• I have reduced the issue to the simplest possible case.

---

Please review the first message of the discussion #9156, it contain a way to reproduce, the expected result, the actual result and even a research with some proposition of fix.

[auvipy]

This issue seems to be a django compatibility issue hence should be considered for further investigation/work towards a fix. thanks for raising

[FraCata00]

If you use:

class MySerializer(serializers.Serializer):
    start_time = serializers.DateTimeField(
        format="%Y-%m-%d %H:%M:%S",
        input_formats=["iso-8601", "%Y-%m-%d %H:%M:%S"],
    )

still work with validation

[RignonNoel]

If you use:

class MySerializer(serializers.Serializer):
    start_time = serializers.DateTimeField(
        format="%Y-%m-%d %H:%M:%S",
        input_formats=["iso-8601", "%Y-%m-%d %H:%M:%S"],
    )

still work with validation

Thanks for the quick fix resolution, a good info to unblock people having this breaking change.

I still think it's really important to fix this issue for two reasons:

1. We really should clarify the definition of ISO-8601 since DRF think using this ISO rule but use a function of Django that is not made for it.
2. It's still a breaking change for people migrating, and there is no trace of this issue on the changelog

[FraCata00]

If you use:

class MySerializer(serializers.Serializer):

start_time = serializers.DateTimeField(

    format="%Y-%m-%d %H:%M:%S",

    input_formats=["iso-8601", "%Y-%m-%d %H:%M:%S"],

)

still work with validation

Thanks for the quick fix resolution, a good info to unblock people having this breaking change.

I still think it's really important to fix this issue for two reasons:

1. We really should clarify the definition of ISO-8601 since DRF think using this ISO rule but use a function of Django that is not made for it.

2. It's still a breaking change for people migrating, and there is no trace of this issue on the changelog

Sure about this
Think that the ISO-8601 and the Django fromisoformat still working good, but not for all formats..
So what about the migration? About Django methods?

[HamaBarhamou]

Hello, I'm currently working on a django ticket here. I plan to join you soon when I finish this tikelt. Assign me the ticket if it's not taken. I'll look into it in my spare time.

[lovelydinosaur]

Please review the first message of the discussion #9156

Now, it seems that 2021-03-13 is valid...

Thanks, no.

We have slightly stricter more careful validation of datetimes here, and the existing behaviour is reasonable.

[kezabelle]

@tomchristie 👋 Hello. Apologies for bumping this, but I wanted to make sure you're fully grokking the reported problem as laid out.¹ Specifically, where you've said:

We have slightly stricter more careful validation of datetimes here

seems like it might be a misunderstanding. I'm sorry if I infer that incorrectly, in which case the rest is all waffle you already knew...

The issue as I understand it² is precisely that the stricter more careful validation of datetimes is being subverted by upstream changes in Django and Python and that previously "invalid" strings are now valid datetimes, at least in the case where the input format is not given and (apparently) defaults to iso-8601.

Specifically, DRF uses the internal parse_datetime function of Django, which had long been regex based, but eventually moved to preferring Python's native datetime.datetime.fromisoformat ... and that method then got augmented with additional parsing capabilities.

Ultimately I think this means that some previously invalid strings become valid datetimes (as far as DRF is concerned) when newer versions of Django (post 4.0+) or Python (post 3.11+) are used in combination; said another way, you've got string parsing dependent on 2 moving dependencies; said another way, the subset of ISO 8601 strings which are (correctly) coerced has expanded, and what's right for Django or Python in terms of behaviour may not be right for DRF, at this juncture.

---

If you wanted to restore the older, extant behaviour I believe you'd be looking at bringing a facsimile of parse_datetime into DRF³, but skipping the changes which moved away from using datetime_re and potentially strictly deciding on what subset is considered valid.⁴

Footnotes

1. I am the original cause 😮‍💨 and am only watching this by virtue of having been drive-by tagged, previously. ↩

2. Which is just a very high level loosey-goosey awareness of this ticket... ↩

3. Potentially also parse_date and parse_time 🤷 ↩

4. e.g. How many microseconds of precision is required, yadda yadda. ↩