Provide an update_display message

[rgbkrk]

The purpose of this message would be to provide an update to a display_data field, without need of full duplex communications.

@ellisonbg suggested that this would make it so that progress bars can be implemented without needing full widgets/comms support. That's exactly the kind of feedback loop I want to be able to provide to users as well.

For me, I want this for informing about any background threads or processes including spark jobs, database queries, and external processes (see ipython/ipython#9969).

Potential approach

On display_data, provide transient field to identify a display_id:

{
    "content": {
      "data": {
        "text/plain": "[==                ]"
      },
      "transient": {
        "display_id": "0000"
      }
    }
}

Followed by an update_display_data:

{
    "content": {
      "data": {
        "text/plain": "[==============    ]"
      },
      "transient": {
        "display_id": "0000"
      }
    }
}

In the notebook, this should also result in storing the last result of the update.

[rgbkrk]

Alternative - top level key:

{
    "content": {
      "data": {
        "text/plain": "[==                ]"
      },
      "display_id": "0000-1111-2222-3333"
    }
}

This makes it so the display_id is not persisted at runtime.

[rgbkrk]

Another suggestion from @minrk: we could have another entry in content for all transient/live data:

{
    "content": {
      "data": {
        "text/plain": "[==                ]"
      },
      "transient": {
        "display_id": "0000-1111-2222-3333"
      }
    }
}

[rgbkrk]

The jupyter js widgets currently use transient data to get bootstrapped on the page by setting the model id in a mimebundle

{
  "application/vnd.jupyter.widget": model_id
}

The transient section seems like a good use case for those as well.

[rgbkrk]

/cc @jasongrout @SylvainCorlay

[minrk]

I like not persisting the display-id, so the top-level key or transient area makes sense to me. These don't really make sense as persisted info (much like it doesn't make sense to persist widget model IDs in the document).

Related details from side conversation:

• update_display would modify the display_data item not just on the page, but in the notebook state

Further questions:

• should this be a new message, or should this be info added to display_data itself?
  What should happen to an update message if the target can't be found (e.g. target output cleared)? Start a new output, or discard as having no destination? This may point to the answer for whether it should be a new message or not.

[rgbkrk]

Now that I'm thinking about model IDs for jupyter widgets, it would be handy to have the transient/live dict/map available in general on all the messages that have mime bundles.

[rgbkrk]

What should happen to an update message if the target can't be found (e.g. target output cleared)? Start a new output, or discard as having no destination? This may point to the answer for whether it should be a new message or not.

Well that one leaves me a bit stumped. I'm not sure what the desired behavior is.

[minrk]

I think it depends a bit on how we define the semantics of update. Do we allow publishing deltas, or is it always replacing the whole thing? If it's always replacing the whole thing, then it seems like the simplest modification would be a regular display_data message with the following modifications to handling:

if display_id is defined:
    if display_id found on the page:
        update match in-place
    else:
        regular display, record display_id for future lookup
else:
    regular display

[rgbkrk]

Ah, I like that. I'm definitely thinking of replacing the whole thing. Doing it on the same display_data by id makes sense to me.

I feel like deltas belong in comms or another message type.

[minrk]

Yup, there's always comms for the complex case.

At the protocol level, we could already have this, because each display_data already has an id: the msg_id. But in practice this is difficult to locate, as it's not typically available in the kernel's APIs.

[minrk]

While this repo only defines protocols, it might be worth considering what IPython APIs could look like for updating an output, since you would have to create and track an ID somehow. Sending raw messages is always an option, but presumably we want to cover at least some use cases without that.

[rgbkrk]

The way I've cheated in the past week for a single thread where I want to update a prior cell:

Which only helps me if I'm ok with only appending.

[rgbkrk]

The reason of course that I can't necessarily get and set header dynamically is that parent_header is an attribute on the ZMQInteractiveShell:

def set_parent(self, parent):
        """Set the parent header for associating output with its triggering input"""
        self.parent_header = parent
        self.displayhook.set_parent(parent)
        self.display_pub.set_parent(parent)
        if hasattr(self, '_data_pub'):
            self.data_pub.set_parent(parent)
        try:
            sys.stdout.set_parent(parent)
        except AttributeError:
            pass
        try:
            sys.stderr.set_parent(parent)
        except AttributeError:
            pass

[rgbkrk]

Thinking on the IPython API for this:

bundle = {
    'text/html': '<b>whoa</b>',
}
IPython.display.display(bundle, raw=True, transient={'id': '1234'})

[takluyver]

I would imagine we would also want a higher level API, which I envisage something like this:

from IPython.display import display, HTML, Updatable
d = Updatable(HTML('<b>whoa</b>'))
display(d)
# ...
d.replace(HTML('<i>awesome</i>'))