optimisations

[martindurant]

• remove abspath from linux/mac make_path_posix and move fast-path to start of long function
• logging in fuse
• give option to don't have instance cache keep instances alive (but still share) using config "weakref_instance_cache"
• allow min/max to cat_file to be negative, like a slice

[ananthsub]

is this a breaking change in the API? in pytorch lightning tests we're seeing this stacktrace now:

kwargs = {}
    def info(self, path, **kwargs):
        if isinstance(path, str):
            path = self._strip_protocol(path)
            out = os.stat(path, follow_symlinks=False)
            link = os.path.islink(path)
            if os.path.isdir(path):
                t = "directory"
            elif os.path.isfile(path):
                t = "file"
            else:
                t = "other"
        else:
            # scandir DirEntry
>           out = path.stat(follow_symlinks=False)
E           TypeError: stat() got an unexpected keyword argument 'follow_symlinks'

example: https://github.com/PyTorchLightning/pytorch-lightning/runs/2766424283?check_suite_focus=true

[martindurant]

No the API did not change, this path is only meant to be called by internal functions. The test is whether the path is a str, and I suspect that the tmpdir fixture used for the test is not actually a str, but something str-like. str(tempdir) in the test would solve it. However, fsspec maybe should have coped with this in the more user-facing isdir/info functions (if my guess about the cause is correct).

[ananthsub]

yes, you're right. tmpdir is a py.path.local object. while we can update our tests to handle with this, does it make sense to update the instance check as well?

[TomAugspurger]

@martindurant what's the purpose of **kwargs here? Should subclasses check that additional kwargs aren't passed?

[martindurant]

It's so that subclasses can pass any extra arguments (like headers) without changing the signature. If the subclass doesn't have any such thing it can do, then kwargs can be ignored.

[TomAugspurger]

without changing the signature.

What's the motivation for not changing the signature? If they subclass accepts additional kwargs, then they can add them. As long as it's a superset of the parent's signature and the function is valid with the defaults then things will work fine.

then kwargs can be ignored.

Functions probably shouldn't silently ignore kwargs, since this leads to difficult to debug problems.

[martindurant]

I guess its an indication to downstream that this is a reasonable place to add extra args, and it enables fsspec to add arguments (like start/end) without immediately braking other implementations for the common case that they are left with default values such as None.