Skip to content

Commit 829bbee

Browse files
estherk15dussault-antoine
authored andcommitted
Add TCP disclaimer to Logs docs (#32724)
* Add TCP disclaimer shortcode * Add tcp disclaimer and remove options * Move tcp endpint to separate section, remove trouleshooting info * Update content/en/agent/logs/proxy.md
1 parent 52cad13 commit 829bbee

File tree

7 files changed

+32
-206
lines changed

7 files changed

+32
-206
lines changed

content/en/agent/logs/log_transport.md

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ further_reading:
2020
---
2121

2222

23-
## Default agent behavior
23+
## Default Agent behavior
2424

2525
For Agent v6.19+/v7.19+, the default transport used for your logs is compressed HTTPS instead of TCP for the previous versions.
2626
When the Agent starts, if log collection is enabled, it runs a HTTPS connectivity test. If successful, then the Agent uses the compressed HTTPS transport, otherwise the Agent falls back to a TCP transport.
@@ -33,7 +33,7 @@ To check which transport is used by the Agent, run the [Agent status command][1]
3333

3434
**Notes**:
3535

36-
* For older Agent versions, TCP transport is used by default. Datadog strongly recommends you to enforce HTTPS transport if you are running v6.14+/v7.14+ and HTTPS compression if you are running v6.16+/v7.16+.
36+
* For older Agent versions, TCP transport is used by default. **Datadog strongly recommends** you to enforce HTTPS transport if you are running v6.14+/v7.14+ and HTTPS compression if you are running v6.16+/v7.16+.
3737
* Always enforce a specific transport (either TCP or HTTPS) when using a proxy to forwards logs to Datadog
3838

3939
## Enforce a specific transport
@@ -111,6 +111,9 @@ When logs are sent through HTTPS, use the same [set of proxy settings][3] as the
111111
{{% tab "TCP" %}}
112112
{{< site-region region="us,eu,us3,us5,ap1,ap2" >}}
113113

114+
{{% logs-tcp-disclaimer %}}
115+
116+
114117
To enforce TCP transport, update the Agent's [main configuration file][1] (`datadog.yaml`) with:
115118

116119
```yaml

content/en/agent/logs/proxy.md

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,9 @@ further_reading:
1313
text: "Collect your traces"
1414
---
1515

16-
{{% site-region region="us3,eu,us5,gov,ap1,ap2" %}}
16+
{{% logs-tcp-disclaimer %}}
17+
18+
{{% site-region region="us3,us5,gov,ap1,ap2" %}}
1719
<div class="alert alert-danger">
1820
TCP is not available for the {{< region-param key="dd_site_name" >}} site. Contact <a href="/help/">support</a> for more information.
1921
</div>

content/en/logs/guide/docker-logs-collection-troubleshooting-guide.md

Lines changed: 0 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -134,30 +134,6 @@ If the Logs Agent status looks like the example in [Check the Agent status](#che
134134
* The required port (10516) for sending logs to Datadog is being blocked.
135135
* Your container is using a different logging driver than the Agent expects.
136136

137-
#### Outbound traffic on port 10516 is blocked
138-
139-
The Datadog Agent sends its logs to Datadog over TCP using port 10516. If that connection is not available, logs fail to be sent and an error is recorded in the `agent.log` file to that effect.
140-
141-
You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For OpenSSL, run the following command:
142-
143-
```shell
144-
openssl s_client -connect intake.logs.datadoghq.com:10516
145-
```
146-
147-
For GnuTLS, run the following command:
148-
149-
```shell
150-
gnutls-cli intake.logs.datadoghq.com:10516
151-
```
152-
153-
And then by sending a log like the following:
154-
155-
```text
156-
<API_KEY> this is a test message
157-
```
158-
159-
If opening the port 10516 is not an option, it is possible to configure the Datadog Agent to send logs through HTTPS by setting the `DD_LOGS_CONFIG_FORCE_USE_HTTP` environment variable to `true`:
160-
161137
#### Your containers are not using the JSON logging driver
162138

163139
Docker's default is the json-file logging driver so the Container Agent tries to read from this first. If your containers are set to use a different logging driver, the Logs Agent indicates that it is able to successfully find your containers but it isn't able to collect their logs. The Container Agent can also be configured to read from the journald logging driver.

content/en/logs/guide/log-collection-troubleshooting-guide.md

Lines changed: 0 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -23,37 +23,6 @@ There are a number of common issues that can get in the way when [sending new lo
2323

2424
Changes in the configuration of the `datadog-agent` won't be taken into account until you have [restarted the Agent][3].
2525

26-
## Outbound traffic on port 10516 is blocked
27-
28-
The Datadog Agent sends its logs to Datadog over TCP using port 10516. If that connection is not available, logs fail to be sent and an error is recorded in the `agent.log` file to that effect.
29-
30-
You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For OpenSSL, run the following command:
31-
32-
```shell
33-
openssl s_client -connect intake.logs.datadoghq.com:10516
34-
```
35-
36-
For GnuTLS, run the following command:
37-
38-
```shell
39-
gnutls-cli intake.logs.datadoghq.com:10516
40-
```
41-
42-
And then by sending a log like the following:
43-
44-
```text
45-
<API_KEY> this is a test message
46-
```
47-
48-
- If opening the port 10516 is not an option, it is possible to configure the Datadog Agent to send logs through HTTPS by adding the following in `datadog.yaml`:
49-
50-
```yaml
51-
logs_config:
52-
force_use_http: true
53-
```
54-
55-
See the [HTTPS log forwarding section][4] for more information.
56-
5726
## Check the status of the Agent
5827

5928
Often, checking the [Agent status command][5] results will help you troubleshoot what is happening.

content/en/logs/log_collection/_index.md

Lines changed: 22 additions & 96 deletions
Original file line numberDiff line numberDiff line change
@@ -154,11 +154,6 @@ Use the [site][13] selector dropdown on the right side of the page to see suppor
154154
| US | HTTPS | `agent-http-intake.logs.datadoghq.com` | 443 | Used by the Agent to send logs in JSON format over HTTPS. See the [Host Agent Log collection documentation][2]. |
155155
| US | HTTPS | `lambda-http-intake.logs.datadoghq.com` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over HTTPS. |
156156
| US | HTTPS | `logs.`{{< region-param key="browser_sdk_endpoint_domain" code="true" >}} | 443 | Used by the Browser SDK to send logs in JSON format over HTTPS. |
157-
| US | TCP | `agent-intake.logs.datadoghq.com` | 10514 | Used by the Agent to send logs without TLS.
158-
| US | TCP and TLS | `agent-intake.logs.datadoghq.com` | 10516 | Used by the Agent to send logs with TLS.
159-
| US | TCP and TLS | `intake.logs.datadoghq.com` | 443 | Used by custom forwarders to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. |
160-
| US | TCP and TLS | `functions-intake.logs.datadoghq.com` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. |
161-
| US | TCP and TLS | `lambda-intake.logs.datadoghq.com` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. |
162157

163158
[1]: /api/latest/logs/#send-logs
164159
[2]: /agent/logs/#send-logs-over-https
@@ -172,9 +167,6 @@ Use the [site][13] selector dropdown on the right side of the page to see suppor
172167
| EU | HTTPS | `agent-http-intake.logs.datadoghq.eu` | 443 | Used by the Agent to send logs in JSON format over HTTPS. See the [Host Agent Log collection documentation][2]. |
173168
| EU | HTTPS | `lambda-http-intake.logs.datadoghq.eu` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over HTTPS. |
174169
| EU | HTTPS | `logs.`{{< region-param key="browser_sdk_endpoint_domain" code="true" >}} | 443 | Used by the Browser SDK to send logs in JSON format over HTTPS. |
175-
| EU | TCP and TLS | `agent-intake.logs.datadoghq.eu` | 443 | Used by the Agent to send logs in protobuf format over an SSL-encrypted TCP connection. |
176-
| EU | TCP and TLS | `functions-intake.logs.datadoghq.eu` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. |
177-
| EU | TCP and TLS | `lambda-intake.logs.datadoghq.eu` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. |
178170

179171
[1]: /api/latest/logs/#send-logs
180172
[2]: /agent/logs/#send-logs-over-https
@@ -252,95 +244,9 @@ Use the [site][13] selector dropdown on the right side of the page to see suppor
252244

253245
### Custom log forwarding
254246

255-
Any custom process or logging library able to forward logs through **TCP** or **HTTP** can be used in conjunction with Datadog Logs.
256-
257-
{{< tabs >}}
258-
{{% tab "HTTP" %}}
259-
260-
You can send logs to Datadog platform over HTTP. Refer to the [Datadog Log HTTP API documentation][1] to get started.
261-
262-
[1]: /api/latest/logs/#send-logs
263-
{{% /tab %}}
264-
{{% tab "TCP" %}}
265-
266-
{{< site-region region="us" >}}
267-
268-
You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For GnuTLS, run the following command:
269-
270-
```shell
271-
gnutls-cli intake.logs.datadoghq.com:10516
272-
```
273-
274-
For OpenSSL, run the following command:
275-
276-
```shell
277-
openssl s_client -connect intake.logs.datadoghq.com:10516
278-
```
279-
280-
You must prefix the log entry with your [Datadog API Key][1] and add a payload.
281-
282-
```
283-
<DATADOG_API_KEY> Log sent directly using TLS
284-
```
285-
286-
Your payload, or `Log sent directly using TLS` as written in the example, can be in raw, Syslog, or JSON format. If your payload is in JSON format, Datadog automatically parses its attributes.
287-
288-
```text
289-
<DATADOG_API_KEY> {"message":"json formatted log", "ddtags":"env:my-env,user:my-user", "ddsource":"my-integration", "hostname":"my-hostname", "service":"my-service"}
290-
```
291-
292-
[1]: /account_management/api-app-keys/#api-keys
293-
294-
{{< /site-region >}}
295-
296-
{{< site-region region="eu" >}}
297-
298-
You can manually test your connection using OpenSSL, GnuTLS, or another SSL/TLS client. For GnuTLS, run the following command:
299-
300-
```shell
301-
gnutls-cli tcp-intake.logs.datadoghq.eu:443
302-
```
303-
304-
For OpenSSL, run the following command:
305-
306-
```shell
307-
openssl s_client -connect tcp-intake.logs.datadoghq.eu:443
308-
```
309-
310-
You must prefix the log entry with your [Datadog API Key][1] and add a payload.
247+
Any custom process or logging library able to forward logs through **HTTP** can be used in conjunction with Datadog Logs.
311248

312-
```
313-
<DATADOG_API_KEY> Log sent directly using TLS
314-
```
315-
316-
Your payload, or `Log sent directly using TLS` as written in the example, can be in raw, Syslog, or JSON format. If your payload is in JSON format, Datadog automatically parses its attributes.
317-
318-
```text
319-
<DATADOG_API_KEY> {"message":"json formatted log", "ddtags":"env:my-env,user:my-user", "ddsource":"my-integration", "hostname":"my-hostname", "service":"my-service"}
320-
```
321-
322-
[1]: /account_management/api-app-keys/#api-keys
323-
324-
{{< /site-region >}}
325-
326-
{{< site-region region="us3" >}}
327-
The TCP endpoint is not recommended for this site. Contact [support][1] for more information.
328-
329-
[1]: /help
330-
{{< /site-region >}}
331-
332-
{{< site-region region="gov,us5,ap1,ap2" >}}
333-
334-
The TCP endpoint is not supported for this site.
335-
336-
[1]: /help
337-
{{< /site-region >}}
338-
339-
340-
[1]: https://app.datadoghq.com/organization-settings/api-keys
341-
[2]: https://app.datadoghq.com/logs/livetail
342-
{{% /tab %}}
343-
{{< /tabs >}}
249+
You can send logs to Datadog platform over HTTP. Refer to the [Datadog Log HTTP API documentation][15] to get started.
344250

345251
**Notes**:
346252

@@ -357,6 +263,24 @@ Log events that do not comply with these limits might be transformed or truncate
357263

358264
There is an additional truncation in fields that applies only to indexed logs: the value is truncated to 75 KiB for the message field and 25 KiB for non-message fields. Datadog still stores the full text, and it remains visible in regular list queries in the Logs Explorer. However, the truncated version will be displayed when performing a grouped query, such as when grouping logs by that truncated field or performing similar operations that display that specific field.
359265

266+
{{% collapse-content title="TCP" level="h3" expanded=false %}}
267+
268+
{{% logs-tcp-disclaimer %}}
269+
270+
271+
| Site | Type | Endpoint | Port | Description |
272+
|------|-------------|---------------------------------------------------------------------------|--------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
273+
| US | TCP | `agent-intake.logs.datadoghq.com` | 10514 | Used by the Agent to send logs without TLS.
274+
| US | TCP and TLS | `agent-intake.logs.datadoghq.com` | 10516 | Used by the Agent to send logs with TLS.
275+
| US | TCP and TLS | `intake.logs.datadoghq.com` | 443 | Used by custom forwarders to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. |
276+
| US | TCP and TLS | `functions-intake.logs.datadoghq.com` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. |
277+
| US | TCP and TLS | `lambda-intake.logs.datadoghq.com` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. |
278+
| EU | TCP and TLS | `agent-intake.logs.datadoghq.eu` | 443 | Used by the Agent to send logs in protobuf format over an SSL-encrypted TCP connection. |
279+
| EU | TCP and TLS | `functions-intake.logs.datadoghq.eu` | 443 | Used by Azure functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. **Note**: This endpoint may be useful with other cloud providers. |
280+
| EU | TCP and TLS | `lambda-intake.logs.datadoghq.eu` | 443 | Used by Lambda functions to send logs in raw, Syslog, or JSON format over an SSL-encrypted TCP connection. |
281+
282+
{{% /collapse-content %}}
283+
360284
### Attributes and tags
361285

362286
Attributes prescribe [logs facets][9], which are used for filtering and searching in Log Explorer. See the dedicated [attributes and aliasing][10] documentation for a list of reserved and standard attributes and to learn how to support a naming convention with logs attributes and aliasing.
@@ -407,3 +331,5 @@ Once logs are collected and ingested, they are available in **Log Explorer**. Lo
407331
[12]: /logs/explore/
408332
[13]: /getting_started/site/
409333
[14]: /logs/log_configuration/pipelines/?tab=date#date-attribute
334+
[15]: /api/latest/logs/#send-logs
335+

content/en/logs/log_collection/csharp.md

Lines changed: 0 additions & 52 deletions
Original file line numberDiff line numberDiff line change
@@ -494,58 +494,6 @@ using (var log = new LoggerConfiguration()
494494
}
495495
```
496496

497-
{{< site-region region="us" >}}
498-
499-
You can also override the default behavior and forward logs in TCP by manually specifying the following required properties: `url`, `port`, `useSSL`, and `useTCP`. Optionally, [specify the `source`, `service`, `host`, and custom tags.][1]
500-
501-
For instance to forward logs to the Datadog US region in TCP you would use the following sink configuration:
502-
503-
```csharp
504-
var config = new DatadogConfiguration(url: "intake.logs.datadoghq.com", port: 10516, useSSL: true, useTCP: true);
505-
using (var log = new LoggerConfiguration()
506-
.WriteTo.DatadogLogs(
507-
"<API_KEY>",
508-
source: "<SOURCE_NAME>",
509-
service: "<SERVICE_NAME>",
510-
host: "<HOST_NAME>",
511-
tags: new string[] {"<TAG_1>:<VALUE_1>", "<TAG_2>:<VALUE_2>"},
512-
configuration: config
513-
)
514-
.CreateLogger())
515-
{
516-
// Some code
517-
}
518-
```
519-
520-
[1]: /logs/log_configuration/attributes_naming_convention/#reserved-attributes
521-
522-
{{< /site-region >}}
523-
{{< site-region region="eu" >}}
524-
525-
You can also override the default behavior and forward logs in TCP by manually specifying the following required properties: `url`, `port`, `useSSL`, and `useTCP`. Optionally, [specify the `source`, `service`, `host`, and custom tags.][1]
526-
527-
For instance to forward logs to the Datadog EU region in TCP you would use the following sink configuration:
528-
529-
```csharp
530-
var config = new DatadogConfiguration(url: "tcp-intake.logs.datadoghq.eu", port: 443, useSSL: true, useTCP: true);
531-
using (var log = new LoggerConfiguration()
532-
.WriteTo.DatadogLogs(
533-
"<API_KEY>",
534-
source: "<SOURCE_NAME>",
535-
service: "<SERVICE_NAME>",
536-
host: "<HOST_NAME>",
537-
tags: new string[] {"<TAG_1>:<VALUE_1>", "<TAG_2>:<VALUE_2>"},
538-
configuration: config
539-
)
540-
.CreateLogger())
541-
{
542-
// Some code
543-
}
544-
```
545-
[1]: /logs/log_configuration/attributes_naming_convention/#reserved-attributes
546-
547-
{{< /site-region >}}
548-
549497
New logs are now directly sent to Datadog.
550498

551499
## Further Reading
Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
<div class="alert alert-warning">TCP log collection is <strong>not supported</strong>. Datadog provides <strong>no delivery or reliability guarantees</strong> when using TCP, and log data may be lost without notice.
2+
For reliable ingestion, use the HTTP intake endpoint, an official Datadog Agent, or forwarder integration instead. For more information, see <a href="/logs/log_collection/?tab=host">Log Collection</a>.</div>

0 commit comments

Comments
 (0)