Skip to content

Commit 02e8455

Browse files
authored
Quickwit service run several services (#1163)
Closes #1160
1 parent 0db764c commit 02e8455

38 files changed

+1082
-709
lines changed

Cargo.lock

Lines changed: 146 additions & 12 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

docs/get-started/installation.md

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ sidebar_position: 2
44
---
55

66
Quickwit compiles to a single binary, we provide different methods to install it.
7-
We notably provide musl builds to provide static binaries with no dependencies.
7+
We notably provide musl builds to provide static binaries with no dependencies.
88

99

1010
## Download
@@ -25,8 +25,8 @@ Checkout all builds on [github](https://github.com/quickwit-oss/quickwit/release
2525

2626
Quickwit depends on the following external libraries to work correctly:
2727
- `libpq`: the Postgres client library.
28-
- `libssl`: the industry defacto cryptography library.
29-
These libraries can be installed on your system using the native package manager.
28+
- `libssl`: the industry defacto cryptography library.
29+
These libraries can be installed on your system using the native package manager.
3030
On Ubuntu for instance, you can install these dependencies using the following command:
3131

3232
```bash
@@ -35,7 +35,7 @@ apt-get -y update && apt-get -y install libpq-dev libssl-dev
3535

3636
:::note
3737

38-
Quickwit static binary packages are also provided as `musl` builds. These packages don't require you to install any external library and can be automatically picked during installation on your system if the required libc version is not present. You can also download and manually install a static binary package.
38+
Quickwit static binary packages are also provided as `musl` builds. These packages don't require you to install any external library and can be automatically picked during installation on your system if the required libc version is not present. You can also download and manually install a static binary package.
3939

4040
:::
4141

@@ -65,14 +65,14 @@ quickwit-{version}
6565
- `config/quickwit.yaml`: is the default configuration file.
6666
- `LICENSE_AGPLv3.0.txt`: the license file.
6767
- `quickwit`: the quickwit executable binary.
68-
- `qwdata/`: the default data directory.
68+
- `qwdata/`: the default data directory.
6969

7070

7171
## Use the docker image
7272

73-
If you use docker, this might be one of the quickest way to get going.
73+
If you use docker, this might be one of the quickest way to get going.
7474
The following command will pull the image from [dockerhub](https://hub.docker.com/r/quickwit/quickwit)
75-
and gets you right in the shell of the running container ready to execute Quickwit commands.
75+
and gets you right in the shell of the running container ready to execute Quickwit commands.
7676
Note that we are also mounting the working directory as volume. This is useful when you already have your dataset ready on your machine and want to work with Quickwit docker image.
7777

7878
```bash
@@ -90,14 +90,14 @@ mkdir data && cd data
9090
curl -o wikipedia_index_config.yaml https://raw.githubusercontent.com/quickwit-oss/quickwit/main/config/tutorials/wikipedia/index-config.yaml
9191
curl -o wiki-articles-10000.json https://quickwit-datasets-public.s3.amazonaws.com/wiki-articles-10000.json
9292

93-
# create, index and search using the container
93+
# create, index and search using the container
9494
docker run -v "$(pwd)":"/quickwit/qwdata" quickwit/quickwit index create --index-config ./qwdata/wikipedia_index_config.yaml
9595

9696
docker run -v "$(pwd)":"/quickwit/qwdata" quickwit/quickwit index ingest --index wikipedia --input-path ./qwdata/wiki-articles-10000.json
9797

9898
docker run -v "$(pwd)":"/quickwit/qwdata" quickwit/quickwit index search --index wikipedia --query "barack obama"
9999

100-
docker run -v "$(pwd)":"/quickwit/qwdata" --expose 7280 -p 7280:7280 quickwit/quickwit service run searcher
100+
docker run -v "$(pwd)":"/quickwit/qwdata" --expose 7280 -p 7280:7280 quickwit/quickwit run --service searcher
101101
```
102102

103103
Now you can make HTTP requests to the searcher service API.

docs/get-started/quickstart.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,7 @@ It should return 10 hits. Now you're ready to serve our search API.
114114
Quickwit provides a search [REST API](../reference/rest-api.md) that can be started using the `service` subcommand.
115115

116116
```bash
117-
./quickwit service run searcher
117+
./quickwit run --service searcher
118118
```
119119

120120
Check it's working with a simple GET request in the browser or via cURL:

docs/guides/add-full-text-search-to-your-olap-db.md

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ sidebar_position: 2
44
---
55

66

7-
This guide will help you add full-text search to a well-known OLAP database, Clickhouse, using the Quickwit search streaming feature. Indeed Quickwit exposes a REST endpoint that streams ids or whatever attributes matching a search query **extremely fast** (up to 50 million in 1 second), and Clickhouse can easily use them with joins queries.
7+
This guide will help you add full-text search to a well-known OLAP database, Clickhouse, using the Quickwit search streaming feature. Indeed Quickwit exposes a REST endpoint that streams ids or whatever attributes matching a search query **extremely fast** (up to 50 million in 1 second), and Clickhouse can easily use them with joins queries.
88

99
We will take the [Github archive dataset](https://www.gharchive.org/), which gathers more than 3 billion Github events: `WatchEvent`, `PullRequestEvent`, `IssuesEvent`... You can dive into this [great analysis](https://ghe.clickhouse.tech/) made by Clickhouse to have a good understanding of the dataset. We also took strong inspiration from this work, and we are very grateful to them for sharing this.
1010

@@ -97,7 +97,7 @@ You can check it's working by using the `search` command and looking for `tantiv
9797
## Start a searcher
9898

9999
```bash
100-
./quickwit service run searcher
100+
./quickwit run --service searcher
101101
```
102102

103103
This command will start an HTTP server with a [REST API](../reference/rest-api.md). We are now
@@ -163,10 +163,10 @@ gunzip -c gh-archive-2021-12.json.gz | clickhouse-client -d gh-archive --query="
163163
Let's check it's working:
164164
```SQL
165165
# Top repositories by stars
166-
SELECT repo_name, count() AS stars
167-
FROM github_events
168-
WHERE event_type = 'WatchEvent'
169-
GROUP BY repo_name
166+
SELECT repo_name, count() AS stars
167+
FROM github_events
168+
WHERE event_type = 'WatchEvent'
169+
GROUP BY repo_name
170170
ORDER BY stars DESC LIMIT 5
171171

172172
┌─repo_name────────────┬─stars─┐
@@ -181,7 +181,7 @@ ORDER BY stars DESC LIMIT 5
181181
### Use Quickwit search inside Clickhouse
182182

183183
Clickhouse has an exciting feature called [URL Table Engine](https://clickhouse.com/docs/en/engines/table-engines/special/url/) that queries data from a remote HTTP/HTTPS server.
184-
This is precisely what we need: by creating a table pointing to Quickwit search stream endpoint, we will fetch ids that match a query from Clickhouse.
184+
This is precisely what we need: by creating a table pointing to Quickwit search stream endpoint, we will fetch ids that match a query from Clickhouse.
185185

186186
```SQL
187187
SELECT count(*) FROM url('http://127.0.0.1:7280/api/v1/gh-archive/search/stream?query=log4j+OR+log4shell&fastField=id&outputFormat=clickHouseRowBinary', RowBinary, 'id UInt64')

docs/guides/tutorial-hdfs-logs-distributed-search-aws-s3.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -163,7 +163,7 @@ Now let's launch a searcher node for this instance.
163163

164164
```bash
165165
# Then start the http server search service.
166-
./quickwit service run searcher --config ./config.yaml
166+
./quickwit run --service searcher --config ./config.yaml
167167
```
168168

169169
You will see in the terminal the confirmation that the instance has created a new cluster. Example of such a log:

0 commit comments

Comments
 (0)