unpin numpy

[keewis]

• follow-up to install numbagg from conda-forge #7415

It seems in a previous PR I "temporarily" pinned numpy to get CI to pass, but then forgot to unpin later and merged it as-is. As a result, we have not been running the main CI with numpy>=1.24 ever since, even though now numpy=1.25 has been around for a while.

[keewis]

The doctests CI fails because unstacking a int array uses np.full_like to create a integer array but fills it with nan (the result of dtypes.get_fill_value(dtype) for int dtypes). Newer numpy versions complain about this by emitting a warning, which then gets converted to an error.

xarray/xarray/core/variable.py

Lines 1870 to 1880 in eceec5f

	data = np.full_like(
	self.data,
	fill_value=fill_value,
	shape=new_shape,
	dtype=dtype,
	)
	# Indexer is a list of lists of locations. Each list is the locations
	# on the new dimension. This is robust to the data being sparse; in that
	# case the destinations will be NaN / zero.
	data[(..., *indexer)] = reordered

In this particular case, the fill value itself will never appear in the unstacked array, so we could just use np.empty_like instead of np.full_like (not sure what to do for sparse, but maybe we don't need to do anything?).

[headtr1ck]

These warnings were getting annoying, thanks!

[headtr1ck]

Unrelated, but noticed again:
Seeing that new Mypy versions create problems maybe we should pin the Mypy version and set up an updater action with a PR?

[keewis]

Seeing that new Mypy versions create problems maybe we should pin the Mypy version

We can do that, but note that the only error that's new is

xarray/core/dataset.py:503: Definition of "__eq__" in base class "DatasetOpsMixin" is incompatible with definition in base class "Mapping"  [misc]

the other ones appear to be numpy:

xarray/tests/test_groupby.py:731: error: Argument 1 to "groupby" of "Dataset" has incompatible type "ndarray[Any, dtype[signedinteger[Any]]]"; expected "Hashable | DataArray | IndexVariable"  [arg-type]
xarray/tests/test_groupby.py:731: note: Following member(s) of "ndarray[Any, dtype[signedinteger[Any]]]" have conflicts:
xarray/tests/test_groupby.py:731: note:     __hash__: expected "Callable[[], int]", got "None"
xarray/tests/test_coding_strings.py:36: error: Unused "type: ignore" comment  [unused-ignore]

Any idea how to fix those?

[headtr1ck]

I don't think those are numpys fault.
We just have to add ndarray to groupbys argument Type list (I think there is already a DataArray Compatible definition somewhere, maybe that works).

Edit: nevermind, numpy Arrays are not supposed to work and mypy is actually correct here. Your ignore is correct. Somehow the older Mypy did not recognize this.

The Hashable None error usually disappears magically when you fix the other errors.

[headtr1ck]

We just have to add ndarray to groupbys argument Type list

There's a explicit check that raises a TypeError, so ignoring the (intentionally) wrong typing in the test seems appropriate? See ca245d6

Yes sry. Saw that too late. You are totally correct

[keewis]

hah, yeah, I also didn't see your edit soon enough.

In any case, that leaves us with the __eq__ error that is new in mypy=1.5

[keewis]

I barely know what I'm doing when it comes to typing, but from what I can tell the reason is a typeshed update in mypy combined with OVERRIDE_TYPESHED in xarray.util.generate_ops not having any effect Edit: maybe not, I totally missed the .pyi file

[keewis]

once tests pass this should be ready for a final review. Edit: see above for the mypy failure

[dcherian]

LGTM