Add interface for defining an ExtensionIndex

[TomAugspurger]

These occur naturally with .groupby(extensionarray) and Seres[extensionarray].value_counts. We should define a public API so that we don't have to convert to object in these cases.

You'd likely end up with an Index with a non-object dtype.

---

more radically this kind of removes the need for all our Index subclasses, aside from MultiIndex. But we can think about that separately from the interface.

I don't think this is blocking for 1.0, but it may not be too much effort.

[jorisvandenbossche]

Given that we already have / will have several example (CategoricalIndex, IntervalIndex, DatetimeIndex, ...) that actually do this, I think it shouldn't be too hard to distil some common things out of those to have an ExtensionIndex?

(maybe could even give some base class for them)

It's true that our "index as container" model indeed doesn't necessarily need the specific classes (similar as Series does not have several classes). But I think that boat has sailed for pandas itself. The main difference is that on Index subclasses, we allow several type-specific methods, something I personally actually like.

[TomAugspurger]

IMO this would be a good roadmap item once #27478 is in.

[Detry322]

@jreback if I took a stab at this in the near future (a week or so) is there any hope of it being included in 1.0?

[jreback]

yes it’s possible