Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions docs/commands.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# RedisAI Commands
RedisAI is a Redis module, and as such it implements several data types and the respective commands to use them.

All of RedisAI's commands are begin with the `AI.` prefix. The following sections describe these commands.
All of RedisAI's commands begin with the `AI.` prefix. The following sections describe these commands.

**Syntax Conventions**

Expand Down Expand Up @@ -365,7 +365,7 @@ def addtwo(a, b):
It can be stored as a RedisAI script using the CPU device with [`redis-cli`](https://redis.io/topics/rediscli) as follows:

```
$ cat addtwo.py | redis-cli -x AI.SCRIPTSET myscript addtwo CPU TAG SOURCE myscript:v0.1
$ cat addtwo.py | redis-cli -x AI.SCRIPTSET myscript addtwo CPU TAG myscript:v0.1 SOURCE
OK
```

Expand Down Expand Up @@ -514,9 +514,9 @@ The **`AI.DAGRUN`** command specifies a direct acyclic graph of operations to ru

It accepts one or more operations, split by the pipe-forward operator (`|>`).

By default, the DAG execution context is local, meaning that loading and persisting tensors should be done explicitly. The user should specify which key tensors to load from keyspace using the `LOAD` keyword, and which command outputs to persist to the keyspace using the `PERSIST` keyspace.
By default, the DAG execution context is local, meaning that tensor keys appearing in the DAG only live in the scope of the command. That is, setting a tensor with `TENSORSET` will store it local memory and not set it to an actual database key. One can refer to that key in subsequent commands within the DAG, but that key won't be visible outside the DAG or to other clients - no keys are open at the database level.

When `PERSIST` is not present, object savings are done locally and kept only during the context of the DAG meaning that no output keys are open.
Loading and persisting tensors from/to keyspace should be done explicitly. The user should specify which key tensors to load from keyspace using the `LOAD` keyword, and which command outputs to persist to the keyspace using the `PERSIST` keyspace.

As an example, if `command 1` sets a tensor, it can be referenced by any further command on the chaining.

Expand Down Expand Up @@ -611,21 +611,21 @@ The following example obtains the previously-run 'myscript' script's runtime sta

```
redis> AI.INFO myscript
1) KEY
1) key
2) "myscript"
3) TYPE
3) type
4) SCRIPT
5) BACKEND
5) backend
6) TORCH
7) DEVICE
7) device
8) CPU
9) DURATION
9) duration
10) (integer) 11391
11) SAMPLES
11) samples
12) (integer) -1
13) CALLS
13) calls
14) (integer) 1
15) ERRORS
15) errors
16) (integer) 0
```

Expand Down
2 changes: 2 additions & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,8 @@ redis-server --loadmodule /usr/lib/redis/modules/redisai.so \
### THREADS_PER_QUEUE
The **THREADS_PER_QUEUE** configuration option controls the number of worker threads allocated to each device's job queue. Multiple threads can be used for executing different independent operations in parallel.

Note that RedisAI maintains one job queue per device (CPU, GPU:0, GPU:1). Each job queue is consumed by THREADS_PER_QUEUE threads.

This option significantly improves the performance of simple, low-effort computation-wise models since there is spare computation cycle available from modern CPUs and hardware accelerators (GPUs, TPUs, ...).

_Expected Value_
Expand Down
6 changes: 2 additions & 4 deletions docs/developer.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,15 @@ The following sections discuss topics relevant to the development of the RedisAI

RedisAI bundles together best-of-breed technologies for delivering stable and fast model serving. To do so, we need to abstract from what each specific DL/ML framework offers and provide common data structures and APIs to the DL/ML domain.


As a way of representing tensor data we've embraced [dlpack](https://github.com/dmlc/dlpack) - a community effort to define a common tensor data structure that can be shared by different frameworks, supported by cuPy, cuDF, DGL, TGL, PyTorch, and MxNet.

**Data Structures**

RedisAI provides the following data structures:

* **Tensor**: represents an n-dimensional array of values
* **Model**: represents a frozen graph by one of the supported DL/ML framework backends
* **Script**: represents a [TorchScript](https://pytorch.org/docs/stable/jit.html)
* **Model**: represents a computation graph by one of the supported DL/ML framework backends
* **Script**: represents a [TorchScript](https://pytorch.org/docs/stable/jit.html) program

## Source code layout

Expand All @@ -33,7 +32,6 @@ of complexity incrementally.

**redisai.c**


This is the entry point of the RedisAI module, responsible for registering the new commands in the Redis server, and containing all command functions to be called. This file is also responsible for exporting of Tensor, Script and Model APIs to other Modules.

**tensor.h**
Expand Down
Binary file added docs/images/graph.pb.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 3 additions & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,12 @@

RedisAI is a Redis module for executing Deep Learning/Machine Learning models and managing their data. Its purpose is being a "workhorse" for model serving, by providing out-of-the-box support for popular DL/ML frameworks and unparalleled performance. RedisAI both simplifies the deployment and serving of graphs by leveraging on Redis' production-proven infrastructure, as well as maximizes computation throughput by adhering to the principle of data locality.

RedisAI is a joint effort between [Redis Labs](https://www.redislabs.com) and [Tensorwerk](https://tensorwerk.com).

## Where Next?
* The [Introduction](intro.md) is the recommended starting point
* The [Quickstart](quickstart.md) page provides information about building, installing and running RedisAI
* The [Commands](commands.md) page is a reference of RedisAI's API
* The [Commands](commands.md) page is a reference of the RedisAI API
* The [Clients](clients.md) page lists RedisAI clients by programming language
* The [Configuration](configuration.md) page explains how to configure RedisAI
* The [Developer](developer.md) page has more information about the design and implementation of the RedisAI module
Expand Down
30 changes: 18 additions & 12 deletions docs/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,10 @@ In broad strokes, RedisAI's looks as follows:
| | | myscript +----->+ Script +---+ +----+-----+ +-->+ ONNXRuntime | | | |
| | | | | +--------+ | ^ | +-------------+ | | |
| | +----------+ | | | | | | |
| | | | | +--------+ | | | +-------------+ | | |
| | | mydag +----->+ DAG +---+ | +-->+ ... | | | |
| | | | | +--------+ | +-------------+ | | |
| | +----------+ ^ +------------------------|-----------------------------+ | |
| | ^ | +--------+ | | | +-------------+ | | |
| | | | + DAG +---+ | +-->+ ... | | | |
| | | | +--------+ | +-------------+ | | |
| | | +------------------------|-----------------------------+ | |
| +--------------|--------------------------|-------------------------------+ |
| v v |
| +--------------+-----------------+ +------------------------------------+ |
Expand Down Expand Up @@ -83,22 +83,22 @@ A **Redis module** is a shared library that can be loaded by the Redis server du
* [Modules published at redis.io](https://redis.io/modules)

### Why RedisAI?
RedisAI bundles together best-of-breed technologies for delivering stable and performant graph serving. Every DL/ML framework ships with a backend for executing the graphs developed by it, and the common practice for serving these is building a simple server.
RedisAI bundles together best-of-breed technologies for delivering stable and performant computation graph serving. Every DL/ML framework ships with a runtime for executing the models developed with it, and the common practice for serving these is building a simple server around them.

RedisAI aims to be that server, saving you from the need of installing the backend you're using and developing a server for it. By itself that does not justify RedisAI's existence so there's more to it. Because RedisAI is implemented as a Redis module it automatically benefits from the server's capabilities: be it Redis' native data types, its robust eco-system of clients, high-availability, persistence, clustering, and Enterprise support.

Because Redis is an in-memory data structure server RedisAI uses it for storing all of its data. The main data type supported by RedisAI is the Tensor that is the standard representation of data in the DL/ML domain. Because tensors are stored memory space of the Redis server they are readily accessible to any of RedisAI's backend libraries at minimal latency.
Because Redis is an in-memory data structure server RedisAI uses it for storing all of its data. The main data type supported by RedisAI is the Tensor that is the standard representation of data in the DL/ML domain. Because tensors are stored in the memory space of the Redis server, they are readily accessible to any of RedisAI's backend libraries at minimal latency.

The locality of data, which is tensor data in adjacency to DL/ML models backends, allows RedisAI to provide optimal performance when serving models. It also makes it a perfect choice for deploying DL/ML models in production and allowing them to be used by any application.

Furthermore, RedisAI is also an optimal testbed for models as it allows the parallel execution of multiple graphs and, in future versions, assessing their respective performance in real-time.
Furthermore, RedisAI is also an optimal testbed for models as it allows the parallel execution of multiple computation graphs and, in future versions, assessing their respective performance in real-time.

#### Data Structures
RedisAI provides the following data structures:

* **Tensor**: represents an n-dimensional array of values
* **Model**: represents a frozen graph by one of the supported DL/ML framework backends
* **Script**: represents a [TorchScript](https://pytorch.org/docs/stable/jit.html)
* **Model**: represents a computation graph by one of the supported DL/ML framework backends
* **Script**: represents a [TorchScript](https://pytorch.org/docs/stable/jit.html) program

#### DL/ML Backends
RedisAI supports the following DL/ML identifiers and respective backend libraries:
Expand Down Expand Up @@ -135,7 +135,7 @@ docker exec -it redisai redis-cli
```

## Using RedisAI Tensors
A **Tensor** is an n-dimensional array and is the standard vehicle for DL/ML data. RedisAI adds to Redis a Tensor data structure that implements the tensor type. Like any datum in Redis, RedisAI's Tensors are identified by key names.
A **Tensor** is an n-dimensional array and is the standard representation for data in DL/ML workloads. RedisAI adds to Redis a Tensor data structure that implements the tensor type. Like any datum in Redis, RedisAI's Tensors are identified by key names.

Creating new RedisAI tensors is done with the [`AI.TENSORSET` command](commands.md#aitensorset). For example, consider the tensor: $\begin{equation*} tA = \begin{bmatrix} 2 \\ 3 \end{bmatrix} \end{equation*}$.

Expand All @@ -154,7 +154,7 @@ Copy the command to your cli and hit the `<ENTER>` on your keyboard to execute i
OK
```

The reply 'OK' means that the operation was successful. We've called the `AI.TENSORSET` command to set the key named 'tA' with the tensor's data, but the name could have been any string value. The `FLOAT` argument specifies the type of values that the tensor stores, and in this case a double-precision floating-point. After the type argument comes the tensor's shape as a list of its dimensions, or just a single dimension of 2.
The reply 'OK' means that the operation was successful. We've called the `AI.TENSORSET` command to set the key named 'tA' with the tensor's data, but the name could have been any string value. The `FLOAT` argument specifies the type of values that the tensor stores, and in this case a single-precision floating-point. After the type argument comes the tensor's shape as a list of its dimensions, or just a single dimension of 2.

The `VALUES` argument tells RedisAI that the tensor's data will be given as a sequence of numeric values and in this case the numbers 2 and 3. This is useful for development purposes and creating small tensors, however for practical purposes the `AI.TENSORSET` command also supports importing data in binary format.

Expand Down Expand Up @@ -188,7 +188,7 @@ A **Model** is a Deep Learning or Machine Learning frozen graph that was generat

Models, like any other Redis and RedisAI data structures, are identified by keys. A Model's key is created using the [`AI.MODELSET` command](commands.md#aimodelset) and requires the graph payload serialized as protobuf for input.

In our examples, we'll use one of the graphs that RedisAI uses in its tests, namely 'graph.pb', which can be downloaded from [here](https://github.com/RedisAI/RedisAI/raw/master/test/test_data/graph.pb).
In our examples, we'll use one of the graphs that RedisAI uses in its tests, namely 'graph.pb', which can be downloaded from [here](https://github.com/RedisAI/RedisAI/raw/master/test/test_data/graph.pb). This graph was created using TensorFlow with [this script](https://github.com/RedisAI/RedisAI/blob/master/test/test_data/tf-minimal.py).

??? info "Downloading 'graph.pb'"
Use a web browser or the command line to download 'graph.pb':
Expand All @@ -197,6 +197,12 @@ In our examples, we'll use one of the graphs that RedisAI uses in its tests, nam
wget https://github.com/RedisAI/RedisAI/raw/master/test/test_data/graph.pb
```

You can view the computation graph using [Netron](https://lutzroeder.github.io/netron/), which supports all frameworks supported by RedisAI.

![Computation graph visualized in Netron](images/graph.pb.png "Computation Graph Visualized in Netron")

This is a great way to inspect a graph and find out node names for inputs and outputs.

redis-cli doesn't provide a way to read files' contents, so to load the model with it we'll use the command line and output pipes:

```
Expand Down