Skip to content
129 changes: 42 additions & 87 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,127 +1,76 @@
<p align="center">
<span">
<img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo.svg#gh-light-mode-only" alt="Parseable" width="500" height="100" />
<img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo-dark.png#gh-dark-mode-only" alt="Parseable" width="500" height="100" />
<img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo.svg#gh-light-mode-only" alt="Parseable" width="400" height="80" />
<img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/logo-dark.png#gh-dark-mode-only" alt="Parseable" width="400" height="80" />
</a>
</p>

<p align="center">
<a href="https://fossunited.org/" target="_blank"><img src="http://fossunited.org/files/fossunited-badge.svg"></a>
<img src="https://img.shields.io/github/commit-activity/m/parseablehq/parseable" alt="commits activity monthly">
<a href="https://launchpass.com/parseable" target="_blank"><img src="https://img.shields.io/badge/join%20slack-parseable-brightgreen.svg" alt="join slack"></a>
<a href="https://github.com/parseablehq/parseable/stargazers" target="_blank"><img src="https://img.shields.io/github/stars/parseablehq/parseable?style=social" alt="Github stars"></a>
<a href="https://twitter.com/parseableio" target="_blank"><img src="https://img.shields.io/twitter/follow/parseableio" alt="Twitter"></a>
</p>

<h4 align="center">
<p> Parseable is an open source log storage and observability platform, built for Kubernetes. </p>
<img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/console.png" />
<a href="https://www.parseable.io/docs/quick-start" target="_blank">Quick Start</a> |
<a href="https://www.parseable.io/docs/introduction" target="_blank">Documentation</a> |
<a href="https://launchpass.com/parseable" target="_blank">Community</a> |
<a href="https://demo.parseable.io" target="_blank">Live Demo</a>
<br>
</h4>

Parseable is a cloud native, log storage and observability platform. Parseable is written in Rust and index free by design. Parseable is available as a single binary / Docker image and can be deployed on a wide range of platforms.
## Why Parseable?

It ingests log data via HTTP POST calls and exposes a query API to search and analyze logs. It is compatible with all the standard logging agents via HTTP output plugins.
As SREs, DevOps deploy and manage more and more cloud native applications, there is a glaring gap in the ecosystem for a developer friendly, cloud native, event & log observability platform. We faced this issue first hand at our jobs, and we've seen several other large scale Kubernetes users relating to this gap.

<img src="https://raw.githubusercontent.com/parseablehq/.github/main/images/console.gif" />
Parseable started because of this _missing_ case. We intend to enrich the ecosystem with a developer friendly, cloud native, event & log observability platform.
## How it works

## Features
Parseable exposes REST API to ingest and query log data. Under the hood, it uses Apache Arrow and Parquet to handle and compress high volume log data. All data is stored in S3 (or compatible systems). Parseable also has a bundled web console to visualize and query log data.

- [x] Highly compressed log data storage with [Parquet](https://parquet.apache.org).
- [x] Use standard SQL for querying log data.
- [x] Auto inferred schema for log streams.
- [x] Dashboard to query the log data.
- [x] Compatible with existing logging agents.
- [x] Scale with scaling up the node.
- [ ] Configurable alerting.
- [ ] Distributed, multi-node cluster.
- [ ] Log data visualization via Parseable UI.
- [ ] Access control for users and groups.
- [ ] Log data retention and export.
- [ ] Kafka plugin to ingest log data.
- [ ] Grafana plugin to visualize log data.
- [ ] Anomaly detection within log data.
Parseable can be deployed in cloud and container based environments in minutes and can scale as the requirements grow.

## Getting Started
#### Key differentiators

### Docker
Parseable docker image is available on [Docker hub](https://hub.docker.com/r/parseable/parseable). Please change the environment variables as relevant.
- Written in Rust. Low CPU & memory footprint, with low latency, high throughput.
- Open data format (Parquet). Complete ownership of data. Wide range of possibilities for data analysis.
- Single binary / container based deployment (including UI). Deploy in minutes if not seconds.
- Indexing free design. Lower CPU and storage overhead. Similar levels of performance as indexing based systems.

```sh
cat << EOF > parseable-env
P_S3_URL=https://minio.parseable.io:9000
P_S3_ACCESS_KEY=minioadmin
P_S3_SECRET_KEY=minioadmin
P_S3_REGION=us-east-1
P_S3_BUCKET=parseable
P_LOCAL_STORAGE=/data
P_USERNAME=parseable
P_PASSWORD=parseable
EOF
## Installing

Docker is the quickest way to experience Parseable on your machine. Run the below command to deploy Parseable in demo mode.

```sh
mkdir -p /tmp/data
docker run \
-p 8000:8000 \
--env-file parseable-env \
-v /tmp/data:/data \
parseable/parseable:latest \
parseable server
parseable server --demo
```

### Kubernetes
Parseable helm chart is available at [Parseable charts repository](https://charts.parseable.io/).
Once this runs successfully, you'll see dashboard at [http://localhost:8000](http://localhost:8000). You can login to the dashboard with `parseable`, `parseable` as the credentials. Please make sure not to post any important data while in demo mode.

```sh
helm repo add parseable https://charts.parseable.io/
helm repo update
kubectl create namespace parseable
helm install parseable parseable/parseable --namespace parseable --set parseable.demo=true
```
For non-demo and other installation options (Kubernetes, bare-metal), please refer to the [documentation](https://www.parseable.io/docs/category/installation).

### Binary
Parseable binary is available on [Github releases](https://github.com/parseablehq/parseable/releases). Please download the latest release for your platform, also make sure to change the environment variables as relevant.
#### Live demo

```sh
export P_S3_URL="https://minio.parseable.io:9000"
export P_S3_ACCESS_KEY="minioadmin"
export P_S3_SECRET_KEY="minioadmin"
export P_S3_REGION="us-east-1"
export P_S3_BUCKET="parseable"
export P_LOCAL_STORAGE="./data"
export P_USERNAME="parseable"
export P_PASSWORD="parseable"
chmod +x parseable
./parseable
```
Instead of installing locally, you can also try out Parseable on our [Demo instance](https://demo.parseable.io). Credentials to login to the dashboard are `parseable` / `parseable`.

<h1></h1>
## Usage

Parseable dashboard is available at [http://localhost:8000](http://localhost:8000). Credentials to login to the dashboard are the values you set in the environment variables.
If you've already deployed Parseable using the above Docker command, use below commands to create stream and post event(s) to the stream. Make sure to replace `<stream-name>` with the name of the stream you want to create and post events (e.g. `my-stream`).

:memo: Parseable is in alpha stage and will evolve over time. There may be breaking changes between releases. Please give us your feedback in [Slack](https://launchpass.com/parseable), or [Issues](https://github.com/parseablehq/parseable/issues/new).

## Using Parseable
`<stream-name>` is the name of the stream you want to create. `<basic-auth-header>` is the basic auth header value generated from username & password of the user you created in the environment variables. You can generate the basic auth header value using the following command.

```sh
echo -n '<user-name>:<password>' | base64
```

### Create a stream
#### Create a stream

```sh
curl --location --request PUT 'http://localhost:8000/api/v1/logstream/<stream-name>' \
--header 'Authorization: Basic <basic-auth-header>'
--header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ=='
```

### Send events to the stream
#### Send events to the stream

```sh
curl --location --request POST 'http://localhost:8000/api/v1/logstream/<stream-name>' \
--header 'X-P-META-meta1: value1' \
--header 'X-P-TAG-tag1: value1' \
--header 'Authorization: Basic <basic-auth-header>' \
--header 'Authorization: Basic cGFyc2VhYmxlOnBhcnNlYWJsZQ==' \
--header 'Content-Type: application/json' \
--data-raw '[
{
Expand All @@ -136,16 +85,22 @@ curl --location --request POST 'http://localhost:8000/api/v1/logstream/<stream-n
]'
```

For complete Parseable API documentation, refer to [Parseable API Ref Docs](https://www.parseable.io/docs/api-reference).
For complete Parseable API documentation, refer to [Parseable API Docs](https://www.parseable.io/docs/category/api).

## Live Demo
## Support

For questions and feedback please feel free to reach out to us on [Slack](https://launchpass.com/parseable). For bugs, please create issue on [GitHub](https://github.com/parseablehq/parseable/issues).

For commercial support and consultation, please reach out to us at [`[email protected]`](mailto:[email protected]).

You can also try out Parseable on our [https://demo.parseable.io](https://demo.parseable.io). Credentials to login to the dashboard are `parseable` / `parseable`.

## Contributing

Refer to the contributing guide [here](https://www.parseable.io/docs/contributing).

### Contributors
#### Contributors

<a href="https://github.com/parseablehq/parseable/graphs/contributors"><img src="https://contrib.rocks/image?repo=parseablehq/parseable" /></a>

#### Supported by

<a href="https://fossunited.org/" target="_blank"><img src="http://fossunited.org/files/fossunited-badge.svg"></a>