Add documentation on building custom components for apps and distributing as packages

_quarto.yml

@@ -106,7 +106,7 @@ website:
       - icon: youtube
         href: https://www.youtube.com/playlist?list=PL9HYL-VRX0oRbLoj3FyL5zeASU5FMDgVe
         aria-label: Shiny YouTube Playlist
-  
+
   sidebar:
     - id: components
       collapse-level: 2
@@ -189,6 +189,10 @@ website:
             - docs/ipywidgets.qmd
             - docs/ui-html.qmd
             - docs/workflow-server.qmd
+        - section: "Extending"
+          contents:
+            - docs/custom-component-one-off.qmd
+            - docs/custom-components-pkg.qmd
         - section: "Framework Comparisons"
           contents:
             - docs/comp-streamlit.qmd

docs/custom-component-one-off.qmd

@@ -0,0 +1,362 @@
+---
+title: "Custom JavaScript component"
+---
+
+In this post, you will learn how to create a custom element and accompanying output binding in Shiny. This is useful if you want to create an output that is not currently in Shiny for your app.
+
+:::{.callout-note}
+
+This post talks about making a one-off component for a given app. If you plan on reusing your component or distributing it for others to use, see the accompanying post on [making a custom component package.](custom-components-pkg.html)
+:::
+
+:::{.callout-warning}
+The code shown here is simplified to get the point across, but before you use it in your own app, you should make sure to add error handling and other features to make it robust.
+:::
+
+# The problem
+
+You found a new table library that you really want to use in your Shiny app. The library is [Tabulator](https://tabulator.info/), which is a JavaScript library for making tables with data. But there's a problem: there's no way to easily use it from a Shiny app. To do this, we'll need to write some Python code that will let us use the library from the Python side of Shiny, and wrap the library's JavaScript code to make it talk to JavaScript side of Shiny.
+
+# The solution
+
+To implement a custom Tabulator element for your app, you'll need to write three things:
+
+1. A JavaScript script that renders the element on the client side using the Tabulator library
+2. An `output_tabulator()` function for placing the element in your app's UI
+3. A `render_tabulator()` decorator for passing table data to the JavaScript code rendering the element on the server side
+
+
+## The JavaScript code
+
+First things first: to use a custom JavaScript library we need to write... some JavaScript.
+
+To do this we will create a new folder called `tabulator/` that has the following structure:
+
+:::{.callout-note}
+This example uses plain JavaScript with no build step. For an example using typescript and with a build-step see the accompanying article on [making a custom component package.](custom-components-pkg.html)
+:::
+
+```
+tabulator/
+  tabulator_esm.min.js
+  tabulator.min.css
+  tableComponent.js
+```
+
+Both `tabulator_esm.min.js` and `tabulator.min.css` are downloaded from [tabulator's website.](https://tabulator.info/docs/5.5/install#sources-download) `tableComponent.js` is the script that we will write that contains the code for rendering the table to our Shiny app.
+
+:::{.callout-note}
+The code in this article will be abbreviated to show the relevant parts. If you want to see the full code, see the [accompanying repo.](https://github.com/posit-dev/pyshiny-output-binding-example)
+:::
+
+
+To create an output binding in Shiny, we create a new instance of the `Shiny.OutputBinding` class.
+
+```{.javascript filename="tableComponent.js"}
+class TabulatorOutputBinding extends Shiny.OutputBinding {
+    // Find element to render in
+    find(scope) { ... }
+
+    // Render output element in the found element
+    renderValue(el, payload) { ... }
+}
+
+// Register the binding
+Shiny.outputBindings.register(
+  new TabulatorOutputBinding(),
+  "shiny-tabulator-output"
+);
+```
+
+This class has two methods that we need to implement: `find()` and `renderValue()`. The `find()` method is used to identify the element that will contain the rendered table. The `renderValue()` method is used to render the table in the element. After making that class we need to register it with Shiny so it can find and send data to instances of our output.
+
+### The `find()` method
+
+
+Now that we have the scaffolding set up we can start by filling in the `find` method. This function is passed a `scope` object, which is a `jQuery` selection and should return the element you wish to render your output into.
+
+```{.javascript filename="tableComponent.js"}
+class TabulatorOutputBinding extends Shiny.OutputBinding {
+    find(scope) {
+        return scope.find(".shiny-tabulator-output");
+    }
+
+    renderValue(el, payload) {...}
+}
+
+Shiny.outputBindings.register(...);
+```
+
+Note that we're using the class `".shiny-tabulator-output"` here to mark the element that we want to render the table in. This is the same class that we will use in our `output_tabulator()` function in our app's server code. You can use any valid CSS selector here, but it's common to use a class name that descibes the output.
+
+### The `renderValue()` method
+
+Next, we fill in the main logic for rendering our table in to the `renderValue` method. This method gets passed two arguments: `el`, which is an [HTMLElement](https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement) identified by our find function, and `payload`, which is the data that the server has provided from the render function (more on this [soon](#the-render_tabulator-decorator)).
+
+```{.javascript filename="tableComponent.js"}
+// Import the Tabulator library
+import { Tabulator } from "./tabulator_esm.min.js";
+
+class TabulatorOutputBinding extends Shiny.OutputBinding {
+    find(scope) { ... }
+
+    renderValue(el, payload) {
+      // Unpack the info we get from the associated render function
+      const { columns, data, type_hints } = payload;
+
+      // Convert the column names to a format that Tabulator expects
+      const columnsDef = columns.map((col, i) => {
+        return {
+          title: col,
+          field: col,
+          hozAlign: type_hints[i] === "numeric" ? "right" : "left",
+        };
+      });
+
+      // Data comes in as a series of rows with each row having as many elements
+      // as there are columns in the data. We need to map this to a series of
+      // objects with keys corresponding to the column names.
+      function zipRowWithColumns(row) {
+        const obj = {};
+        row.forEach((val, i) => {
+          obj[columns[i]] = val;
+        });
+        return obj;
+      }
+
+      // Instantiate a new Tabulator table in the element.
+      // This will also destroy any existing table in the element
+      // so we don't have to worry about adding and removing tables.
+      new Tabulator(el, {
+        data: data.map(zipRowWithColumns),
+        layout: "fitColumns",
+        columns: columnsDef,
+      });
+    }
+}
+
+Shiny.outputBindings.register(...);
+```
+
+The implementation of this function is not terribly important and draws directly from the [tabulator docs](https://tabulator.info/docs/5.5/quickstart).
+
+What matters is that we take our data, transform it in some way, and then instantiate our table with the `new Tabulator(el, {...})` call. In this case we take data in the form of the rows of a passed data frame, the column names, and the types of those columns (this is determined by [the render decorator](#the-render_tabulator-decorator)), and construct a js object in the form of `data = [{col1: foo1, col2: bar1, ...}, {col1: foo2, col2: bar2, ...}]`. We also combine the column names and types to create the `columnsDef` object that Tabulator expects.
+
+Don't worry too much about following this particular format because it will change depending on the component that you are wrapping.
+
+:::{.callout-note}
+
+This code relies on the `Shiny` object existing in the JavaScript context, but you may want to wrap all the above code in an `if (Shiny)` statement so it only runs if the `Shiny` object exists. This is useful if you're writing a component that might be used in a non-Shiny context because your code won't error out trying to access the non-existant `Shiny` variable and break the document.
+
+```{.javascript filename="tableComponent.js"}
+if (Shiny) {
+    class TabulatorOutputBinding extends Shiny.OutputBinding { ... }
+
+    Shiny.outputBindings.register(...);
+}
+```
+:::
+
+To see the full JavaScript code, see `tabulator/tableComponent.js` in the [accompanying repo.](https://github.com/posit-dev/pyshiny-output-binding-example/blob/main/tabulator/tableComponent.js)
+
+## The `output_tabulator()` function
+
+Next we need an HTML element to target with our JavaScript code. When we set up [the find method](#the-find-method) for our binding, we chose the class `shiny-tabulator-output` as the mark of a tabualtor output, so we need to add that class. We also need to allow the user to set the ID of the element so that Shiny knows which element to target with which output. By wrapping the `id` argument in `resolve_id()` we make sure it will work in the context of modules. We'll also add a height argument so that the user can set the height of the table.
+
+```{.python filename="app.py"}
+from shiny import ui, App
+from shiny.module import resolve_id
+
+from htmltools import HTMLDependency
+
+tabulator_dep = HTMLDependency(
+    "tabulator",
+    "5.5.2",
+    source={"subdir": "tabulator"},
+    script={"src": "tableComponent.js", "type": "module"},
+    stylesheet={"href": "tabulator.min.css"},
+    all_files=True,
+)
+
+def output_tabulator(id, height="200px"):
+    return ui.div(
+        tabulator_dep,
+        # Use resolve_id so that our component will work in a module
+        id=resolve_id(id),
+        class_="shiny-tabulator-output",
+        style=f"height: {height}",
+    )
+```
+
+
+:::{.callout-note}
+We use the `HTMLDependency` function to bind up the assets needed for tabulator that we made in the previous step to make sure th at they're included in our app whenever the `output_tabulator()` function is called (but not more than once).
+
+Note the use of `all_files=True` here. This makes it so we can do the ESM import of the Tabulator library. Otherwise `tabulator_esm.min.js` would not be hosted and the JS library wouldn't be able to find it.
+:::
+
+Now, the `output_tabulator()` function can be called anywhere we want to render a table in our app.
+
+## The `render_tabulator()` decorator
+
+Now we've got the client-side logic finished, we need to write a custom render decorator that sends our data into the component.
+
+A render function's job is to take the result of calling the decorated function, transform it into the format our client-side code wants (in many cases this may be as simple as just returning the object unchanged), and then returning that client-side friendly data which will be passed to our client's `renderValue()` method.
+
+To do this we can leverage some tools provided by Shiny in the `shiny.render.transformer` subpackage.
+
+```{.python filename="app.py"}
+from shiny.render.transformer import (
+    output_transformer,
+    resolve_value_fn,
+    TransformerMetadata,
+    ValueFn,
+)
+
+
+@output_transformer
+async def render_tabulator(
+    _meta: TransformerMetadata,
+    _fn: ValueFn[pd.DataFrame | None],
+):
+    res = await resolve_value_fn(_fn)
+    if res is None:
+        return None
+
+    if not isinstance(res, pd.DataFrame):
+        # Throw an error if the value is not a dataframe
+        raise TypeError(f"Expected a pandas.DataFrame, got {type(res)}. ")
+
+    # Get data from dataframe as a list of lists where each inner list is a
+    # row, column names as array of strings and types of each column as an
+    # array of strings
+    return {
+        "data": res.values.tolist(),
+        "columns": res.columns.tolist(),
+        "type_hints": res.dtypes.astype(str).tolist(),
+    }
+```
+:::{.callout-note}
+In the code above we use types so that we can get some type checking in our IDE, but these are not required.
+:::
+
+The `output_transformer` decorator is a decorator factory (it's a decorator that creates decorators!) that takes a function that returns a dictionary of data to be passed to the client side. The function that it decorates is passed two arguments: `_meta` and `_fn`.
+
+`_meta` is a dictionary of metadata about the function that is being decorated. We don't use it in our example.
+
+`_fn` is the decorated function, i.e. the function that goes below the `@render_tabulator()` decorator in your app's server code. In this case we are expecting the function to return either a pandas dataframe or `None`.
+
+
+```{.python}
+...
+    res = await resolve_value_fn(_fn)
+...
+```
+
+`resolve_value_fn()` is a helper provided in `shiny.render.transformer` for resolving the value of a function that may or may not be async.
+
+
+:::{.callout-note}
+# Why an asynchronous function?
+It is required by Shiny that the output decorator function be `async`. This allows users of the render bindings to provide either synchronous or asynchronous functions to the decorator. This ensures that the function will work whether or not the end user has defined their render function asynchronously.
+
+If you don't need any async behavior you can simply write your function as you would a standard synchronous function after `await`ing `resolve_value_fn()`.
+:::
+
+
+Next, we check to make sure that the value returned by the function is a dataframe. If it's not, we throw an error. This is not strictly required, but is a best practice.
+
+```python
+...
+if not isinstance(res, pd.DataFrame):
+    # Throw an error if the value is not a dataframe
+    raise TypeError(f"Expected a pandas.DataFrame, got {type(res)}. ")
+...
+```
+
+Finally, we return a dictionary of data that we want to pass to the client side. In this case we return the data as a list of lists, the column names as an array of strings, and the types of each column as an array of strings using methods provided by pandas.
+
+```python
+...
+return {
+    "data": res.values.tolist(),
+    "columns": res.columns.tolist(),
+    "type_hints": res.dtypes.astype(str).tolist(),
+}
+...
+```
+
+This returned value is then what gets sent to the client side and is available in the `payload` argument of the `renderValue()` method of our `TabulatorOutputBinding` class.
+
+# The result
+
+Now we have all the components neccesary to use our tabulator output component. Here's an app that uses it to render some number of rows of the indomitable `mtcars` dataset.
+
+```{.python filename="app.py"}
+from shiny import ui, App
+from pathlib import Path
+import pandas as pd
+
+# Code for the custom output
+...
+
+# App code
+app_ui = ui.page_fluid(
+    ui.input_slider("n", "Number of rows to show", 1, 20, 10),
+    output_tabulator("tabulatorTable"),
+)
+
+
+def server(input, output, session):
+    @render_tabulator
+    def tabulatorTable():
+        return pd.read_csv(Path(__file__).parent / "mtcars.csv").head(input.n())
+
+
+app = App(app_ui, server)
+```
+
+Which results in the following app:
+
+
+
+<style>
+/*
+  Co-opt the sourceCode class for nice dropshadow to
+  separate screenshot from background
+*/
+
+.sourceCode:has(.app-screenshot) {
+  padding:0;
+
+  & > p {
+      margin:0;
+  }
+
+  .quarto-figure {
+    margin-bottom: 0;
+  }
+
+  img {
+    margin: 0;
+  }
+
+  figcaption {
+      margin-inline:1rem;
+      margin-block: 0.25rem;
+      font-style: italic;
+  }
+}
+
+.sourceCode:has(.app-screenshot.pad-top) img {
+    margin-block-start: 0.5rem;
+}
+</style>
+
+
+:::{.sourceCode}
+![Our app with custom Tabulator component.](./custom-components-result.png){fig-alt="Screenshot of resulting app" class="app-screenshot"}
+:::
+
+
+To see the full app script, see `app.py` in the accompanying repo for this post [here.](https://github.com/posit-dev/pyshiny-output-binding-example/blob/main/app.py)