[BUG] TraceUpdater fails to update graphs in Tabs after switching tab

[mhangaard]

Thank you to the developers for creating plotly-resampler - I find it really impressive and useful.
Here is a bug which happens, when I am using it with trace-updater, dcc.Tabs and pattern-matching callbacks.

🖍️ Description
When updating a FigureResampler Graph with pattern-matching callbacks using TraceUpdater, when the dynamic graphs are inside Tabs and after zooming on a figure, switching tab, zooming on the other figure, then on switching tab back to the previous one I get the error

TraceUpdater: no graphs with ID="38102ebd-1ebf-4822-b34a-da6a16bde5aa" found"

Furthermore, Dash in debug mode prints

(This error originated from the built-in JavaScript code that runs Dash apps. Click to see the full stack trace or open your browser's console.)
SyntaxError: TraceUpdater: no graphs with ID="38102ebd-1ebf-4822-b34a-da6a16bde5aa" found

    at o.value (http://127.0.0.1:9023/_dash-component-suites/trace_updater/trace_updater.v0_0_9_1m1699522145.min.js:2:76852)

    at finishClassComponent (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:17295:33)

    at updateClassComponent (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:17245:26)

    at beginWork (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:18755:18)

    at HTMLUnknownElement.callCallback (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:182:16)

    at Object.invokeGuardedCallbackDev (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:231:18)

    at invokeGuardedCallback (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:286:33)

    at beginWork$1 (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:23338:9)

    at performUnitOfWork (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:22289:14)

    at workLoopSync (http://127.0.0.1:9023/_dash-component-suites/dash/deps/react-dom@16.v2_14_1m1699522151.14.0.js:22265:24)

🔍 Reproducing the bug
I adapted 03_minimal_cache_dynamic.py to show the graphs inside tabs

"""Minimal dynamic dash app example.
...
"""

from typing import List
from uuid import uuid4

import numpy as np
import plotly.graph_objects as go
from dash import MATCH, Input, Output, State, dcc, html, no_update
from dash_extensions.enrich import (
    DashProxy,
    Serverside,
    ServersideOutputTransform,
    Trigger,
    TriggerTransform,
)
from trace_updater import TraceUpdater

from plotly_resampler import FigureResampler
from plotly_resampler.aggregation import MinMaxLTTB

# Data that will be used for the plotly-resampler figures
x = np.arange(2_000_000)
noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000

# --------------------------------------Globals ---------------------------------------
app = DashProxy(__name__, transforms=[ServersideOutputTransform(), TriggerTransform()])

app.layout = html.Div(
    [
        html.Div(children=[html.Button("Add Chart", id="add-chart", n_clicks=0)]),
        html.Div(
            children=[
                dcc.Tabs(
                    id="container",
                    children=[]
                ),
            ]
        )
    ]
)


# ------------------------------------ DASH logic -------------------------------------
# This method adds the needed components to the front-end, but does not yet contain the
# FigureResampler graph construction logic.
@app.callback(
    Output("container", "children"),
    Input("add-chart", "n_clicks"),
    State("container", "children"),
    prevent_initial_call=True,
)
def add_graph_div(n_clicks: int, div_children: List[dcc.Tab]):
    uid = str(uuid4())
    new_child = dcc.Tab(
        label=uid,
        children=[
            # The graph and its needed components to serialize and update efficiently
            # Note: we also add a dcc.Store component, which will be used to link the
            #       server side cached FigureResampler object
            dcc.Graph(id={"type": "dynamic-graph", "index": uid}, figure=go.Figure()),
            dcc.Loading(dcc.Store(id={"type": "store", "index": uid})),
            TraceUpdater(id={"type": "dynamic-updater", "index": uid}, gdID=f"{uid}"),
            # This dcc.Interval components makes sure that the `construct_display_graph`
            # callback is fired once after these components are added to the session
            # its front-end
            dcc.Interval(
                id={"type": "interval", "index": uid}, max_intervals=1, interval=1
            ),
        ],
    )
    div_children.append(new_child)
    return div_children


# This method constructs the FigureResampler graph and caches it on the server side
@app.callback(
    Output({"type": "dynamic-graph", "index": MATCH}, "figure"),
    Output({"type": "store", "index": MATCH}, "data"),
    State("add-chart", "n_clicks"),
    Trigger({"type": "interval", "index": MATCH}, "n_intervals"),
    prevent_initial_call=True,
)
def construct_display_graph(n_clicks) -> FigureResampler:
    fig = FigureResampler(
        go.Figure(),
        default_n_shown_samples=2_000,
        default_downsampler=MinMaxLTTB(parallel=True),
    )

    # Figure construction logic based on a state variable, in our case n_clicks
    sigma = n_clicks * 1e-6
    fig.add_trace(dict(name="log"), hf_x=x, hf_y=noisy_sin * (1 - sigma) ** x)
    fig.add_trace(dict(name="exp"), hf_x=x, hf_y=noisy_sin * (1 + sigma) ** x)
    fig.update_layout(title=f"<b>graph - {n_clicks}</b>", title_x=0.5)

    return fig, Serverside(fig)


@app.callback(
    Output({"type": "dynamic-updater", "index": MATCH}, "updateData"),
    Input({"type": "dynamic-graph", "index": MATCH}, "relayoutData"),
    State({"type": "store", "index": MATCH}, "data"),
    prevent_initial_call=True,
    memoize=True,
)
def update_fig(relayoutdata: dict, fig: FigureResampler):
    if fig is not None:
        return fig.construct_update_data(relayoutdata)
    return no_update


# --------------------------------- Running the app ---------------------------------
if __name__ == "__main__":
    app.run_server(debug=True, port=9023)

🔧 Expected behavior
I expected Resampler to work independently for each figure and trigger a resample callback for that figure when zooming, autoscaling, and resetting. I expected this to keep working without errors, when switching tab, and returning to a previous tab.

📸 Screenshots
Dash app with Graphs in Tabs. I labeled the tabs by the uid.

The error from Dash in debug mode

Environment information: (please complete the following information)

• OS: Ubuntu 22.04.3 LTS (from Windows with WSL2)
• Python environment:
  • Python version: 3.11.2
  • plotly-resampler version: 0.9.1
  • trace-updater version: 0.0.9.1

$ pip list
Package              Version
-------------------- ------------
ansi2html            1.8.0
blinker              1.7.0
cachelib             0.9.0
certifi              2023.7.22
charset-normalizer   3.3.2
click                8.1.7
dash                 2.14.1
dash-core-components 2.0.0
dash-extensions      1.0.4
dash-html-components 2.0.0
dash-table           5.0.0
dataclass-wizard     0.22.2
EditorConfig         0.12.3
Flask                2.3.3
Flask-Caching        2.0.2
idna                 3.4
importlib-metadata   6.8.0
itsdangerous         2.1.2
Jinja2               3.1.2
jsbeautifier         1.14.11
MarkupSafe           2.1.3
more-itertools       9.1.0
nest-asyncio         1.5.8
numpy                1.26.1
orjson               3.9.10
packaging            23.2
pandas               2.1.2
pip                  23.3.1
plotly               5.18.0
plotly-resampler     0.9.1
python-dateutil      2.8.2
pytz                 2023.3.post1
requests             2.31.0
retrying             1.3.4
setuptools           65.5.0
six                  1.16.0
tenacity             8.2.3
trace-updater        0.0.9.1
tsdownsample         0.1.2
typing_extensions    4.8.0
tzdata               2023.3
urllib3              2.0.7
Werkzeug             3.0.1
zipp                 3.17.0

[jonasvdd]

Hi @mhangaard,

Thank you for submitting this issue (and sorry for the delayed reply)!

To provide some background, traceUpdater was developed during a period when Dash did not yet contain the Patch feature for partial property update component.

Given this, TraceUpdater has become somewhat redundant and could be effectively replaced with Patch. However, this would imply a huge pull-request (rewriting docs, examples, tests), for which I currently do not have the bandwidth.

Regarding your issue, I think it can be overcome by utilizing a Patch component instead of traceUpdater (the latter performs partial ID matches).

A small thing I can do now, is providing a minimal dash example without TraceUpdater, but which uses patch. (I will try to do this today).

Kind regards,
Jonas

[jonasvdd]

The full example below is a rewrite of 03_cache_dynamic, and as you can see very little has changed!

@app.callback(
    Output({"type": "dynamic-graph", "index": MATCH}, "figure", allow_duplicate=True),
    Input({"type": "dynamic-graph", "index": MATCH}, "relayoutData"),
    State({"type": "store", "index": MATCH}, "data"),
    prevent_initial_call=True,
)
def update_fig(relayoutdata: dict, fig: FigureResampler):
    if fig is not None:
        patched_figure = Patch()  # create patch
        for trace in fig.construct_update_data(relayoutdata)[1:]:  # skip first item as it contains the relayout
            trace_index = trace.pop('index')  # the index of the corresponding trace 
            patched_figure['data'][trace_index] = trace  # add all the other data to patch
        return patched_figure
    return no_update

full example:

from typing import List
from uuid import uuid4

import numpy as np
import plotly.graph_objects as go
from dash import MATCH, Input, Output, State, dcc, html, no_update, Patch
from dash_extensions.enrich import (
    DashProxy,
    Serverside,
    ServersideOutputTransform,
    Trigger,
    TriggerTransform,
)

from plotly_resampler import FigureResampler
from plotly_resampler.aggregation import MinMaxLTTB

# Data that will be used for the plotly-resampler figures
x = np.arange(2_000_000)
noisy_sin = (3 + np.sin(x / 200) + np.random.randn(len(x)) / 10) * x / 1_000

# --------------------------------------Globals ---------------------------------------
app = DashProxy(__name__, transforms=[ServersideOutputTransform(), TriggerTransform()])

app.layout = html.Div(
    [
        html.Div(children=[html.Button("Add Chart", id="add-chart", n_clicks=0)]),
        html.Div(id="container", children=[]),
    ]
)


# ------------------------------------ DASH logic -------------------------------------
# This method adds the needed components to the front-end, but does not yet contain the
# FigureResampler graph construction logic.
@app.callback(
    Output("container", "children"),
    Input("add-chart", "n_clicks"),
    State("container", "children"),
    prevent_initial_call=True,
)
def add_graph_div(n_clicks: int, div_children: List[html.Div]):
    uid = str(uuid4())
    # fmt: off
    new_child = html.Div(
        children=[
            dcc.Graph(id={"type": "dynamic-graph", "index": uid}, figure=go.Figure()),
            dcc.Loading(dcc.Store(id={"type": "store", "index": uid})),
            dcc.Interval(id={"type": "interval", "index": uid}, max_intervals=1, interval=1),
        ],
    )
    # fmt: on
    div_children.append(new_child)
    return div_children


# This method constructs the FigureResampler graph and caches it on the server side
@app.callback(
    Output({"type": "dynamic-graph", "index": MATCH}, "figure", allow_duplicate=True),
    Output({"type": "store", "index": MATCH}, "data"),
    State("add-chart", "n_clicks"),
    Trigger({"type": "interval", "index": MATCH}, "n_intervals"),
    prevent_initial_call=True,
)
def construct_display_graph(n_clicks) -> FigureResampler:
    fig = FigureResampler(
        go.Figure(),
        default_n_shown_samples=2_000,
        default_downsampler=MinMaxLTTB(parallel=True),
    )

    # Figure construction logic based on a state variable, in our case n_clicks
    sigma = n_clicks * 1e-6
    fig.add_trace(dict(name="log"), hf_x=x, hf_y=noisy_sin * (1 - sigma) ** x)
    fig.add_trace(dict(name="exp"), hf_x=x, hf_y=noisy_sin * (1 + sigma) ** x)
    fig.update_layout(title=f"<b>graph - {n_clicks}</b>", title_x=0.5)

    return fig, Serverside(fig)


@app.callback(
    Output({"type": "dynamic-graph", "index": MATCH}, "figure", allow_duplicate=True),
    Input({"type": "dynamic-graph", "index": MATCH}, "relayoutData"),
    State({"type": "store", "index": MATCH}, "data"),
    prevent_initial_call=True,
    memoize=True,
)
def update_fig(relayoutdata: dict, fig: FigureResampler):
    if fig is not None:
        patched_figure = Patch()
        for trace in fig.construct_update_data(relayoutdata)[1:]:
            # print(trace)
            trace_index = trace.pop('index')
            patched_figure['data'][trace_index] = trace
        return patched_figure
    return no_update


# --------------------------------- Running the app ---------------------------------
if __name__ == "__main__":
    app.run_server(debug=True, port=9023, use_reloader=False)

[mhangaard]

Brilliant! Thank you @jonasvdd. It is working well with patch updates.

[jonasvdd]

You're welcome, glad it solved your problem!

I think this issue still is a highly relevant case to omit the TraceUpdater component in the (near) future.

[SW4T400]

FYI: I have tried to use this examples idea in my code and when you double click a chart the construct_update gives a "no_update" object which then gives:

TypeError: 'NoUpdate' object is not subscriptable on fig.construct_update_data(relayoutdata)[1:]

[mhangaard]

Yes, you have to check for no_update

from dash import Patch, no_update    
from dash.exceptions import PreventUpdate

...

traces = fig.construct_update_data(relayout_data)
if traces == no_update:
    raise PreventUpdate

for trace in traces[1:]:

...

[jonasvdd]

@mhangaard @SW4T400

FYI - i'm in the progress of creating a PR that removes traceupdater from plotly-resampler, see #281

[SW4T400]

@mhangaard @SW4T400

FYI - i'm in the progress of creating a PR that removes traceupdater from plotly-resampler, see #281

@jonasvdd I think I have actually found an issue:

In dash example 2 of your refactoring for "patch" branch I have added:

´´´
n = 5000 # reduce x and noisy_sin to < 1000 length => no resampling
x2 = x[::n]
y2=noisy_sin[::n]

and in the plot callback

fig.add_trace(go.Scattergl(name="exp"), hf_x=x2, hf_y=y2 * 1.000002**x2)
´´´

When I zoom in and then autoscale, the trace with lower than n_out points still contains all its datapoints which are outside of the current zoomlevel in relayoutdata and hence the plot autoscales to this!:

Are you aware of this?