diff --git a/README.md b/README.md index 580dd65..54b7a95 100644 --- a/README.md +++ b/README.md @@ -1,52 +1,213 @@ # json-log-viewer -[![Maven Central](https://img.shields.io/maven-central/v/ru.d10xa/json-log-viewer_2.12.svg?label=Maven%20Central%20(Scala%202.12))](https://search.maven.org/artifact/ru.d10xa/json-log-viewer_2.12) [![Maven Central](https://img.shields.io/maven-central/v/ru.d10xa/json-log-viewer_3.svg?label=Maven%20Central%20(Scala%203))](https://search.maven.org/artifact/ru.d10xa/json-log-viewer_3) +## Introduction -The `json-log-viewer` converts JSON logs to a human-readable -format via stdin and offers a Scala.js browser version, -streamlining log analysis for developers and system administrators. +`json-log-viewer` is a versatile tool designed to simplify the analysis +of JSON logs for developers and system administrators. +Whether you work in the command line or prefer a browser-based interface, +`json-log-viewer` provides powerful features for filtering, combining, and visualizing log streams. -[DEMO](https://d10xa.ru/json-log-viewer/) +The primary goal of `json-log-viewer` is to transform raw JSON logs into a human-readable format, +making them significantly easier to interpret and process. +With support for advanced filtering, dynamic configuration updates, multi-source input, +and streamlined configuration management, +this tool offers a comprehensive solution for log analysis across various environments. -![screenshot.png](screenshot.png) +You can also try the browser-based version online: [DEMO](https://your-demo-url.com). -## install +### Example Output +Below is an example of formatted JSON logs in the command-line interface: -``` +![Example Output](screenshot.png) + +## Key Features + +- **Human-Readable Log Output** + Effortlessly transform raw JSON logs into a structured, easy-to-read format. + +- **Flexible Interfaces** + Use the command-line utility or a browser-based application for visualization. + +- **Multi-Source Input** + Combine the outputs of multiple commands into a single unified log stream. + +- **Advanced Filtering** + - Apply regular expressions to extract relevant log entries. + - Use SQL-like queries to filter and query JSON fields. + +- **YAML Configuration** + Define input streams and configurations using YAML files. + +- **Dynamic Configuration Updates** + Hot-reload filters and configurations without restarting the tool. + +- **stdin Support** + Process logs directly from the standard input for real-time analysis. + +- **Integration with k9s** + Seamlessly integrate with the k9s Kubernetes CLI tool to visualize logs directly within k9s. + +## Installation + +### Requirements + +- **Coursier** (for installation and dependency management) + +### Install with Coursier + +Install `json-log-viewer` globally using `coursier`: + +```bash coursier install json-log-viewer --channel https://git.io/JvV0g ``` -## build jvm version +After installation, ensure the tool is available: -``` -sbt stage +```bash +json-log-viewer --help ``` -## run jvm version +## Usage -``` -cat log.txt | ./json-log-viewer/jvm/target/universal/stage/bin/json-log-viewer -``` +### Quick Start +1. Install `json-log-viewer` using Coursier. +2. Pass logs to the tool via stdin: + ```bash + cat log.txt | json-log-viewer + ``` +3. Apply filters directly from the command line: + ```bash + cat log.txt | json-log-viewer --filter "level = 'ERROR'" + ``` -## build js version +### SQL Filtering -``` -sbt fullLinkJS -``` +`json-log-viewer` supports SQL-like filtering for JSON fields, allowing precise log analysis. +You can use comparison and logical operations. + +#### Supported Syntax +- **Comparison operators**: + - `=`: Equal + - `!=`: Not equal + - `LIKE`: Matches with patterns (`%` for wildcard) + - `NOT LIKE`: Negates pattern matching +- **Logical operators**: + - `AND`: Logical AND + - `OR`: Logical OR + - `()` for grouping expressions +- **Literals and Identifiers**: + - Identifiers represent JSON keys (e.g., `level`). + - Literals are strings enclosed in single quotes (e.g., `'ERROR'`). + +#### Examples + +1. Filter logs where `level` is `'ERROR'`: + ```bash + cat log.txt | json-log-viewer --filter "level = 'ERROR'" + ``` +2. Match logs where message contains the word "timeout": + ```bash + cat log.txt | json-log-viewer --filter "message LIKE '%timeout%'" + ``` +3. Combine conditions: + ```bash + cat log.txt | json-log-viewer --filter "level = 'ERROR' AND message LIKE '%timeout%'" + ``` +4. Exclude logs with specific patterns: + ```bash + cat log.txt | json-log-viewer --filter "message NOT LIKE '%timeout%'" + ``` +5. Group conditions with parentheses: + ```bash + cat log.txt | json-log-viewer --filter "(level = 'ERROR' OR level = 'WARN') AND message LIKE '%connection%'" + ``` + +## Configuration + +`json-log-viewer` supports defining input streams, filters, and other settings using a YAML configuration file. + +### YAML Configuration Structure -## run js version +A configuration file consists of one or more **feeds**. +Each feed represents a log source and can have the following attributes: +- **name** (optional): A descriptive name for the feed. +- **commands** (required): A list of shell commands that produce log output. +- **inlineInput** (optional): Direct input as a string instead of executing commands. +- **filter** (optional): SQL-like filter expression for processing logs (e.g., `level = 'ERROR'`). +- **formatIn** (optional): Input log format. Supported values: + - `json` (default). + - `logfmt`. +- **rawInclude** and **rawExclude** (optional): Lists of regular expressions to include or exclude from processing. +### Example Configuration File + +```yaml +feeds: + - name: "application-1-logs" + commands: + - cat log1.txt + filter: | + level = 'ERROR' AND message LIKE '%timeout%' + formatIn: json + rawInclude: + - "ERROR" + rawExclude: + - "DEBUG" + - name: "application-2-logs" + commands: + - cat log2.txt + filter: | + message NOT LIKE '%heartbeat%' + formatIn: logfmt ``` -cat log.txt | node ./json-log-viewer/js/target/scala-3.3.1/json-log-viewer-opt/main.js + +#### Running with a Configuration File + +To use a YAML configuration file, pass its path with the --config-file option: + +```bash +json-log-viewer --config-file json-log-viewer.yml ``` -# frontend-laminar +## Command-Line Arguments + +`json-log-viewer` also supports direct configuration via command-line arguments: + +### Supported Options -```~fastLinkJS::webpack``` +- **--filter**: Apply SQL-like filters directly to logs. + ```bash + cat log.txt | json-log-viewer --filter "level = 'ERROR'" + ``` -# k9s plugin +- **--config-file**: Specify the path to a YAML configuration file. + ```bash + cat log.txt | json-log-viewer --config-file json-log-viewer.yml + ``` + +- **--format-in**: Specify the input log format (supported formats: json, logfmt). + ```bash + cat log.txt | json-log-viewer --format-in logfmt + ``` + +- **--format-out**: Specify the output format (supported formats: pretty, raw). + ```bash + cat log.txt | json-log-viewer --format-out raw + ``` + +- **--timestamp-after** and **--timestamp-before**: Filter logs by a specific time range. + ```bash + cat log.txt | json-log-viewer --timestamp-after 2024-01-01T00:00:00Z --timestamp-before 2024-01-31T23:59:59Z + ``` + +- **--timestamp-field**: Specify the field name for timestamps (default: @timestamp). + ```bash + json-log-viewer --timestamp-field time + ``` + +## k9s Plugin Integrate json-log-viewer with k9s to view formatted JSON logs directly within the k9s interface. @@ -56,7 +217,7 @@ For the plugin to work correctly, the following must be installed on your system 2. **json-log-viewer** (optional) - If already installed, the plugin will use it directly; otherwise, it will fall back to using `coursier` to launch it. ### Installation -Add the following to your k9s plugin file +Add the following to your k9s plugin file (usually located at ~/.k9s/plugins.yaml or, on macOS, check the plugin path with `k9s info`): ```yaml @@ -85,9 +246,64 @@ plugins: fi ``` -## k9s plugin usage +### k9s plugin usage 1. Install json-log-viewer 2. Launch k9s 3. Select a Pod or Container 4. Press Ctrl+L to view logs formatted by json-log-viewer + + +## Development + +This section provides instructions for building and running +both the JVM and JavaScript versions of `json-log-viewer`. +It also includes notes for working on the `frontend-laminar` module. + +### Prerequisites +Ensure you have the following installed on your system: +- Java 11+ (for building the JVM version) +- Node.js 23+ (for building the JS version) +- sbt (Scala Build Tool) + +### Building the JVM Version +To build the JVM version of the project, use the command: + +```bash +sbt stage +``` + +This compiles the code and prepares the executable under the `jvm/target/universal/stage/bin/` directory. + +### Running the JVM Version +Run the application with: +```bash +cat log.txt | ./json-log-viewer/jvm/target/universal/stage/bin/json-log-viewer +``` + +Replace `log.txt` with the path to your JSON log file. The output will be displayed in a human-readable format. + +### Building the JavaScript Version + +To build the JavaScript version, you can use one of the following options: + +1. **Optimized Build**: Use the command: + ```bash + sbt fullLinkJS + ``` + This generates a production-ready JavaScript file located at: + `frontend-laminar/target/scala-3.6.2/frontend-laminar-opt/main.js` +2. **Fast Development Build**: Use the command: + ```bash + sbt fastLinkJS + ``` + This generates a faster, less optimized build located at: + `frontend-laminar/target/scala-3.6.2/frontend-laminar-fastopt/main.js` + +Choose the appropriate option based on your needs: +- Use `fullLinkJS` for production or when you need a fully optimized bundle. +- Use `fastLinkJS` for local development with faster build times. + +## License + +This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for full details.