Skip to content

Revamp Dask Array creation logic to be fully blockwise #105

@gjoseph92

Description

@gjoseph92

When dask/dask#7417 gets in, it may both break the current dask-array-creation hacks, and open the door for a much, much simpler approach: just use da.from_array.

We didn't use da.from_array originally and went through all the current rigamarole because from_array generates a low-level graph, which can be enormous (and slow) for the large datasets we load. But once from_array uses Blockwise, it will be far simpler and more efficient.

We'll just switch to having an array-like class that wraps the asset table and other parameters, and whose __getitem__ basically calls fetch_raster_window. However, it's likely worth thinking about just combining all this into the Reader protocol in some way.

This will also make it easier to do #62 (and may even inadvertently solve it).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions