Skip to content

Conversation

@debnathshoham
Copy link
Contributor

@debnathshoham debnathshoham commented Aug 28, 2021

#PR
In [3]: %time index = pd.to_datetime(np.arange(1_000_000).astype("uint64"), unit="s", utc=True)
CPU times: user 20.6 ms, sys: 19.9 ms, total: 40.5 ms
Wall time: 39.2 ms

In [4]: %time index = pd.to_datetime(np.arange(1_000_000).astype("uint32"), unit="s", utc=True)
CPU times: user 16.3 ms, sys: 6.58 ms, total: 22.8 ms
Wall time: 21.2 ms

#master
In [3]: %time index = pd.to_datetime(np.arange(1_000_000).astype("uint64"), unit="s", utc=True)
CPU times: user 12.6 s, sys: 44.3 ms, total: 12.7 s
Wall time: 12.7 s

In [4]: %time index = pd.to_datetime(np.arange(1_000_000).astype("uint32"), unit="s", utc=True)
CPU times: user 5.88 s, sys: 21.2 ms, total: 5.9 s
Wall time: 5.92 s

@simonjayhawkins simonjayhawkins added Performance Memory or execution speed performance Datetime Datetime data dtype labels Aug 29, 2021
@jreback jreback added this to the 1.4 milestone Aug 30, 2021
@jreback jreback merged commit 303fc9a into pandas-dev:master Aug 30, 2021
@jreback
Copy link
Contributor

jreback commented Aug 30, 2021

thanks @debnathshoham very nice

@debnathshoham debnathshoham deleted the gh42606 branch August 30, 2021 13:48
feefladder pushed a commit to feefladder/pandas that referenced this pull request Sep 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Datetime Datetime data dtype Performance Memory or execution speed performance

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BUG: to_datetime very slow with unsigned ints for unix seconds

3 participants