Skip to content

feat: enable auto-population of missing metadata in logs and printing structured logs to stdout #654

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Jan 18, 2022
Merged

feat: enable auto-population of missing metadata in logs and printing structured logs to stdout #654

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
c1322b2
feat: add support for metadata auto-population and redirection to STD…
minherz Jan 16, 2022
cceb48d
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jan 16, 2022
a233647
chore: fix auto-population monitored resource (#657)
minherz Jan 16, 2022
a629b3e
chore: update readme with auto-population and redirect to stdout info…
minherz Jan 18, 2022
2acfdf2
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jan 18, 2022
964896b
Merge branch 'main' into structured_logging
minherz Jan 18, 2022
22ee67c
🦉 Updates from OwlBot
gcf-owl-bot[bot] Jan 18, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 39 additions & 3 deletions .readme-partials.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,16 @@ custom_content: |
<!-- Optional: defaults to "ERROR" -->
<flushLevel>WARNING</flushLevel>

<!-- Optional: auto detects on App Engine Flex, Standard, GCE and GKE, defaults to "global". See supported resource types -->
<!-- Optional: defaults to ASYNC -->
<writeSynchronicity>SYNC</writeSynchronicity>

<!-- Optional: defaults to true -->
<autoPopulateMetadata>false</autoPopulateMetadata>

<!-- Optional: defaults to false -->
<redirectToStdout>true</redirectToStdout>

<!-- Optional: auto detects on App Engine Flex, Standard, GCE and GKE, defaults to "global". -->
<resourceType></resourceType>

<!-- Optional: defaults to the default credentials of the environment -->
Expand All @@ -39,7 +48,7 @@ custom_content: |
</configuration>
```

In your code :
In your code:

```java
import org.slf4j.Logger;
Expand All @@ -57,4 +66,31 @@ custom_content: |
testLogger.log("test");
}
}
```
Copy link
Contributor

@losalex losalex Jan 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a newline?

```

### Populate log entries with metadata

The library provides multiple ways to enrich log entries with additional information.
You can select one or more ways to customize log entries that will be written into Cloud Logging.

* Manually update a log entry using [`LoggingEnhancer`]
(https://github.com/googleapis/java-logging/blob/main/google-cloud-logging/src/main/java/com/google/cloud/logging/.
* Use [`ILoggingEvent`](https://logback.qos.ch/apidocs/ch/qos/logback/classic/spi/ILoggingEvent.html) to update a log entry with [`LoggingEventEnhancer`]
(https://github.com/googleapis/java-logging-logback/blob/main/src/main/java/com/google/cloud/logging/logback/LoggingEventEnhancer.java).
* Enable [auto-population](https://github.com/googleapis/java-logging#auto-population-of-log-entrys-metadata) of the `LogEntry` metadata using the `autoPopulateMetadata` configuration flag.

#### Optimize log ingestion

By default, the appender will ingest log entries asynchronously by calling Logging API.
Multiple calls may be aggregated before being sent to improve use of API quota and bandwidth.
You can set the `writeSynchronicity` configuration flag to `SYNC` if they want to ingest log entries synchronously.
Note that configuring synchronous ingestion will probably result in performance penalties to your applications.
If you plan to deploy your application in one of Google Cloud managed environments (e.g. Cloud Run, Cloud Function or App Engine),
you can leverage the support provided by the implicit logging agent and the [structured logging](https://cloud.google.com/logging/docs/structured-logging) feature.
To use it, set the `redirectToStdout` configuration flag to `true`.
This flag instructs the appender to print the log entries to `stdout` instead of ingesting them using Logging API.
The log entries are printed using the [structured logging Json format](https://cloud.google.com/logging/docs/structured-logging#special-payload-fields).
In result, the logging agent will be responsible for ingesting the logs to Logging API.
Note that using the structured logging Json format you cannot control the log name where the logs will be ingested.
The logs will be ingested into the project that hosts the environment where your application is running.
The configuration `logDestinationProjectId` will be ignored.
41 changes: 39 additions & 2 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,16 @@ See [Logback filters](https://logback.qos.ch/manual/filters.html#thresholdFilter
<!-- Optional: defaults to "ERROR" -->
<flushLevel>WARNING</flushLevel>

<!-- Optional: auto detects on App Engine Flex, Standard, GCE and GKE, defaults to "global". See supported resource types -->
<!-- Optional: defaults to ASYNC -->
Copy link
Contributor

@losalex losalex Jan 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you supposing to edit README?

<writeSynchronicity>SYNC</writeSynchronicity>

<!-- Optional: defaults to true -->
<autoPopulateMetadata>false</autoPopulateMetadata>

<!-- Optional: defaults to false -->
<redirectToStdout>true</redirectToStdout>

<!-- Optional: auto detects on App Engine Flex, Standard, GCE and GKE, defaults to "global". -->
<resourceType></resourceType>

<!-- Optional: defaults to the default credentials of the environment -->
Expand All @@ -110,7 +119,7 @@ See [Logback filters](https://logback.qos.ch/manual/filters.html#thresholdFilter
</configuration>
```

In your code :
In your code:

```java
import org.slf4j.Logger;
Expand All @@ -130,6 +139,34 @@ public class TestLogger {
}
```

### Populate log entries with metadata

The library provides multiple ways to enrich log entries with additional information.
You can select one or more ways to customize log entries that will be written into Cloud Logging.

* Manually update a log entry using [`LoggingEnhancer`]
(https://github.com/googleapis/java-logging/blob/main/google-cloud-logging/src/main/java/com/google/cloud/logging/.
* Use [`ILoggingEvent`](https://logback.qos.ch/apidocs/ch/qos/logback/classic/spi/ILoggingEvent.html) to update a log entry with [`LoggingEventEnhancer`]
(https://github.com/googleapis/java-logging-logback/blob/main/src/main/java/com/google/cloud/logging/logback/LoggingEventEnhancer.java).
* Enable [auto-population](https://github.com/googleapis/java-logging#auto-population-of-log-entrys-metadata) of the `LogEntry` metadata using the `autoPopulateMetadata` configuration flag.

#### Optimize log ingestion

By default, the appender will ingest log entries asynchronously by calling Logging API.
Multiple calls may be aggregated before being sent to improve use of API quota and bandwidth.
You can set the `writeSynchronicity` configuration flag to `SYNC` if they want to ingest log entries synchronously.
Note that configuring synchronous ingestion will probably result in performance penalties to your applications.
If you plan to deploy your application in one of Google Cloud managed environments (e.g. Cloud Run, Cloud Function or App Engine),
you can leverage the support provided by the implicit logging agent and the [structured logging](https://cloud.google.com/logging/docs/structured-logging) feature.
To use it, set the `redirectToStdout` configuration flag to `true`.
This flag instructs the appender to print the log entries to `stdout` instead of ingesting them using Logging API.
The log entries are printed using the [structured logging Json format](https://cloud.google.com/logging/docs/structured-logging#special-payload-fields).
In result, the logging agent will be responsible for ingesting the logs to Logging API.
Note that using the structured logging Json format you cannot control the log name where the logs will be ingested.
The logs will be ingested into the project that hosts the environment where your application is running.
The configuration `logDestinationProjectId` will be ignored.




## Samples
Expand Down
Loading