Dashboard plot sizing issue with `bokeh=3`

[bryevdv]

ref: bokeh/bokeh#12503

It looks like there are some changes in the upcoming Bokeh major version bump that will need changes on the Dask side. cc @jrbourbeau what do you want to do here? If possible I think it would be advised to set a <3.0 version cap for now.

[jrbourbeau]

Thanks for the heads up @bryevdv. We can easily set an upper bound if needed. When's the 3.0 release scheduled to come out? We might be able to patch things up here before then

[bryevdv]

@jrbourbeau Ostensibly next Monday but if you need a little more time to get another release out with a cap, we can wait a little longer.

[jrbourbeau]

Rerunning CI in #5648 to see what failures look like. Is there an downstream CI build over in bokeh that has failures?

Our normal release schedule has a new release coming out next Friday, though we could also push out something earlier if needed. Setting the <3.0 upper bound with a new release is straightforward for conda-forge but (I think?) impossible for PyPI

[bryevdv]

@jrbourbeau there is one in https://github.com/bokeh/bokeh/actions/runs/3299130418/jobs/5442278509

this shows the Panel rename issue, but then there is also an issue with passing explicit id values which is no longer supported. We could potentially add deprecated WARs for those on our end, but there are also JS issues with CDS column length inconsistencies, so that will require some actual investigation and updates.

impossible for PyPI

If so, I'd recommend a programmatic check in dask with a meaningful message ("requires version > 2.1.1 and < 3.0" etc)

[bryevdv]

Our normal release schedule has a new release coming out next Friday, though we could also push out something earlier if needed.

I will just propose to our team to postpone one week. I think if you can also get a release out slightly earlier without too much hassle, that couldn't hurt, though.

[bryevdv]

BTW @jrbourbeau if you have a script or notebook that exercises the dashboard and would be useful for smoke / acceptance I'd love to check it in somewhere on our end so it's available for anyone to test with. I've been using this but something more substantial would be better think

from dask.distributed import Client, progress
import dask
import dask.dataframe as dd
from sklearn.linear_model import LinearRegression

client = Client(threads_per_worker=4,
                n_workers=2, memory_limit='2GB')

df = dask.datasets.timeseries()

def train(partition):
    est = LinearRegression()
    est.fit(partition[['x']].values, partition.y.values)
    return est

# client.dashboard_link

# df[['x', 'y']].resample('1h').mean().head()
# df.groupby('name').apply(train, meta=object).compute()

[gabicca]

Hi, when can we expect a release of distributed that accounts for the latest Bokeh version changes? We are getting the following error in our pipeline:

/usr/local/lib/python3.9/site-packages/distributed/client.py:4565: in get_task_stream
    return self.sync(
/usr/local/lib/python3.9/site-packages/distributed/utils.py:339: in sync
    return sync(
/usr/local/lib/python3.9/site-packages/distributed/utils.py:406: in sync
    raise exc.with_traceback(tb)
/usr/local/lib/python3.9/site-packages/distributed/utils.py:379: in f
    result = yield future
/usr/local/lib/python3.9/site-packages/tornado/gen.py:762: in run
    value = future.result()
/usr/local/lib/python3.9/site-packages/distributed/client.py:4589: in _get_task_stream
    from distributed.dashboard.components.scheduler import task_stream_figure
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
    from __future__ import annotations
    
    import logging
    import math
    import operator
    import os
    from collections import OrderedDict, defaultdict
    from collections.abc import Iterable
    from datetime import datetime
    from numbers import Number
    from typing import Any, TypeVar
    
    import numpy as np
    from bokeh.core.properties import value, without_property_validation
    from bokeh.io import curdoc
    from bokeh.layouts import column, row
>   from bokeh.models import (
        AdaptiveTicker,
        Arrow,
        BasicTicker,
        BoxSelectTool,
        BoxZoomTool,
        CDSView,
        ColorBar,
        ColumnDataSource,
        CustomJSHover,
        DataRange1d,
        FactorRange,
        GroupFilter,
        HoverTool,
        HTMLTemplateFormatter,
        NumberFormatter,
        NumeralTickFormatter,
        OpenURL,
        Panel,
        PanTool,
        Range1d,
        ResetTool,
        Tabs,
        TapTool,
        Title,
        VeeHead,
        WheelZoomTool,
    )
E   ImportError: cannot import name 'Panel' from 'bokeh.models' (/usr/local/lib/python3.9/site-packages/bokeh/models/__init__.py)
/usr/local/lib/python3.9/site-packages/distributed/dashboard/components/scheduler.py:17: ImportError

This is with bokeh==3.0.1 and distributed==2022.10.2

[jrbourbeau]

@gabicca the latest distributed=2022.11.0 release has bokeh=3 compatibility updates. This will fix the ImporError you're seeing, though there are some known issues with dashboard plot sizing

[jrbourbeau]

Here's an example of the dashboard plot sizing issue (from @mrocklin)

The vertical spacing seems to be gone