BUG: fix astype conversion string -> float

[mlondschien]

• Is a step towards closing BUG: Collection of inconsistencies in .astype conversions #37626
• tests added / passed
• passes black pandas
• passes git diff upstream/master -u -- "*.py" | flake8 --diff
• whatsnew entry

Fixes the string -> float conversion as discussed in #37626 in presence of pd.NA. I used the same approach as used in string -> Int64. Note that differently to the Int64 case, we return a numpy.ndarray, not a pd.array.

I added a test that compares pd.Series. Replacing this with pd.array the tm.assert_numpy_array_equal assertion will raise, as expected is of type FloatingArray. IIUC this is a new feature of pandas 1.2.0. Should we return this in .astype("float") by default?

[jreback]

pls add a whatsnew note as well (I don't think this actually closes the original as you mentioned many cases, but that's ok)

[jreback]

can you use the nulls_fixture (though may have to skip for pd.NaT)

[mlondschien]

Can you expand what for? IIUC pd.Series(["1.1", null], dtype="string") should be equal for null in pd.NA, np.nan, pd.NaT (works), None. This behaviour should be tested somewhere else and adding the fixture would just result in repeating the same test four times.

[ivanovmg]

Thank you for the PR!
Please see the comments below.

Would you consider fixing the similar issue with astype("int") conversion as well in a separate PR?

[mlondschien]

Sure.

Would you consider fixing the similar issue with astype("int") conversion as well in a separate PR?

Can you expand on the issue and, if not obvious, with the expected behaviour?

[ivanovmg]

Sure.

Would you consider fixing the similar issue with astype("int") conversion as well in a separate PR?

Can you expand on the issue and, if not obvious, with the expected behaviour?

Similar to float.

In [1]: import pandas as pd
In [2]: ser = pd.Series([1, pd.NA, 2], dtype="string")
In [3]: ser.astype("int")
...
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NAType'

[mlondschien]

In [1]: import pandas as pd
In [2]: ser = pd.Series([1, pd.NA, 2], dtype="string")
In [3]: ser.astype("int")
...
TypeError: int() argument must be a string, a bytes-like object or a number, not 'NAType'

Hmm. As far as I understood, int (the non-extension array dtype) does not support missings. See:

In [1]: import numpy as np
In [2]: import pandas as pd
In [3]: pd.Series([1.0, np.nan], dtype="float").astype("int")
ValueError: Cannot convert non-finite values (NA or inf) to integer

However, using the nullable extension array dtype Int64, the conversion works:

In [4]: pd.Series(["1", pd.NA], dtype="string").astype("Int64")
Out[4]: 
0       1
1    <NA>
dtype: Int64

Would you prefer if above (string with pd.NA to int) raised a ValueError instead?

Edit: Were you possibly referring toobject -> Int64 as in pd.Series(["1", "2"]).astype("Int64")?

Unrelated: Any idea what's up with the failing tests?

[jreback]

ping on green

[mlondschien]

@jreback Could you have another look?

[jreback]

lgtm. @jorisvandenbossche if any comments.

[jreback]

thanks @mlondschien