subclasses of pathlib.PurePosixPath never call __init__ or __new__

[conchylicultor]

BPO	41109
Nosy	@pitrou, @websurfer5, @Conchylicultor, @Louis-Vincent
PRs	bpo-41109: subclasses of pathlib.Path and pathlib.PurePath now call the subclass's __init__() and __new__() functions when returning new objects #21920

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2020-06-25.02:51:23.231>
labels = ['type-bug', '3.8', '3.9', '3.10', '3.7', 'library']
title = 'subclasses of pathlib.PurePosixPath never call __init__ or __new__'
updated_at = <Date 2020-08-19.06:37:57.142>
user = 'https://github.com/conchylicultor'

bugs.python.org fields:

activity = <Date 2020-08-19.06:37:57.142>
actor = 'Jeffrey.Kintscher'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2020-06-25.02:51:23.231>
creator = 'conchylicultor'
dependencies = []
files = []
hgrepos = []
issue_num = 41109
keywords = ['patch']
message_count = 12.0
messages = ['372297', '372298', '375030', '375031', '375041', '375047', '375162', '375332', '375333', '375334', '375364', '375638']
nosy_count = 4.0
nosy_names = ['pitrou', 'Jeffrey.Kintscher', 'conchylicultor', 'louis-vincent.boudre']
pr_nums = ['21920']
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue41109'
versions = ['Python 3.6', 'Python 3.7', 'Python 3.8', 'Python 3.9', 'Python 3.10']

[conchylicultor]

I have a subclass GithubPath of PurePosixPath.

class GithubPath(pathlib.PurePosixPath):

  def __new__(cls, *args, **kwargs):
    print('New')
    return super().__new__(cls, *args, **kwargs)

  def __init__(self, *args, **kwargs):
    print('Init')
    super().__init__()

Calling child.parent create a new GithubPath but without ever calling new nor init. So my subclass is never notified it is created.

p = GithubPath()  # Print "New", "Init"

p.parent  # Create a new GithubPath but bypass the constructors

The reason seems to be that parent calls _from_parts which create a new object through object.__new__(cls):

cpython/Lib/pathlib.py

Line 689 in cf18c9e

def _from_parts(cls, args, init=True):

A hack is to subclass _init but it seems hacky as it relies on internal implementation detail.

[conchylicultor]

Note that this likely affect all methods which returns new Path by calling _from_parts or _from_parsed_parts, like .absolute, .resolve,...

[websurfer5]

The workaround is to override _init(), which I agree is not desirable.

This is the relevant code in PurePath, which is the super class of PurePosixPath:

    @classmethod
    def _from_parsed_parts(cls, drv, root, parts, init=True):
        self = object.__new__(cls)
        self._drv = drv
        self._root = root
        self._parts = parts
        if init:
            self._init()
        return self

...

    def _init(self):
        # Overridden in concrete Path
        pass

To me, the clean way to get the desired behavior seems like it would be to have _init() call self.__init__().

    def _init(self):
        # Overridden in concrete Path
        self.__init__()

This fixes p.parent, but GithubPath() ends up calling GithubPath.__init__() twice – the first time by PurePath.__new__() calling PurePath._init() and the second time by the GithubPath object creation.

[websurfer5]

Clarification:

PurePath.__new__() calls PurePath._from_parts(), which then calls PurePath._init()

[conchylicultor]

Before solving this issue, I think it would be best to think on a more generic solution on how to make Pathlib more extensible. Related to: https://discuss.python.org/t/make-pathlib-extensible/3428

For instance, if childs created with p.parent(), p / 'subdir' need to forward some state (e.g. RemotePath(path, password=, user=)).

Rather than __init__, maybe there should be some __post_init__ like dataclasses.