Skip to content

DataArray.chunk() from sparse array produces malformed dask array #3191

@crusaderky

Description

@crusaderky

#3117 by @nvictus introduces support for sparse in plain xarray.
dask already supports it.

Running with:

  • xarray git head
  • dask 2.2.0
  • numpy 1.16.4
  • sparse 0.7.0
  • NUMPY_EXPERIMENTAL_ARRAY_FUNCTION=1
>>> import numpy, sparse, xarray, dask.array
>>> s = sparse.COO(numpy.array([1, 2]))         
>>> da1 =  dask.array.from_array(s)
>>> da1._meta
<COO: shape=(0,), dtype=int64, nnz=0, fill_value=0>
>>> da1.compute()
<COO: shape=(2,), dtype=int64, nnz=2, fill_value=0>
>>> da2 = xarray.DataArray(s).chunk().data
>>> da2._meta                                                                                                                                                                                                      
array([], dtype=int64)  # Wrong
>>> da2.compute()
RuntimeError: Cannot convert a sparse array to dense automatically. To manually densify, use the todense method.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions