Skip to content

Commit e6fdc70

Browse files
gsvigruhacswatt
andauthored
[MLOB-4457] Update links/images for the new evaluations experience (#32741)
* update links * update image * draft * applying edits * restore old image and just make a new one --------- Co-authored-by: cecilia saixue watt <[email protected]>
1 parent 029c883 commit e6fdc70

File tree

5 files changed

+14
-14
lines changed

5 files changed

+14
-14
lines changed

content/en/llm_observability/evaluations/_index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ aliases:
88

99
## Overview
1010

11-
LLM Observability offers several ways to support evaluations. They can be configured by navigating to [**AI Observability > Settings > Evaluations**][8].
11+
LLM Observability offers several ways to support evaluations. They can be configured by navigating to [**AI Observability > Evaluations**][8].
1212

1313
### Custom LLM-as-a-judge evaluations
1414

@@ -47,4 +47,4 @@ In addition to evaluating the input and output of LLM requests, agents, workflow
4747
[5]: /llm_observability/evaluations/submit_nemo_evaluations
4848
[6]: /security/sensitive_data_scanner/
4949
[7]: /account_management/rbac/permissions/#llm-observability
50-
[8]: https://app.datadoghq.com/llm/settings/evaluations
50+
[8]: https://app.datadoghq.com/llm/evaluations

content/en/llm_observability/evaluations/custom_llm_as_a_judge_evaluations.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ Learn more about the [compatibility requirements][6].
3030
### Configure the prompt
3131

3232
1. In Datadog, navigate to the LLM Observability [Evaluations page][1]. Select **Create Evaluation**, then select **Create your own**.
33-
{{< img src="llm_observability/evaluations/custom_llm_judge_1.png" alt="The LLM Observability Evaluations page with the Create Evaluation side panel opened. The first item, 'Create your own,' is selected. " style="width:100%;" >}}
33+
{{< img src="llm_observability/evaluations/custom_llm_judge_1-2.png" alt="The LLM Observability Evaluations page with the Create Evaluation side panel opened. The first item, 'Create your own,' is selected. " style="width:100%;" >}}
3434

3535
2. Provide a clear, descriptive **evaluation name** (for example, `factuality-check` or `tone-eval`). You can use this name when querying evaluation results. The name must be unique within your application.
3636

@@ -234,7 +234,9 @@ Refine your prompt and schema until outputs are consistent and interpretable.
234234

235235
## Viewing and using results
236236

237-
After you save your evaluation, Datadog automatically runs your evaluation on targeted spans. Results are available across LLM Observability in near-real-time. You can find your custom LLM-as-a-judge results for a specific span in the **Evaluations** tab, alongside other evaluations.
237+
After you **Save and Publish** your evaluation, Datadog automatically runs your evaluation on targeted spans. Alternatively, you can **Save as Draft** and edit or enable your evaluation later.
238+
239+
Results are available across LLM Observability in near-real-time for published evaluations. You can find your custom LLM-as-a-judge results for a specific span in the **Evaluations** tab, alongside other evaluations.
238240

239241
{{< img src="llm_observability/evaluations/custom_llm_judge_3-2.png" alt="The Evaluations tab of a trace, displaying custom evaluation results alongside managed evaluations." style="width:100%;" >}}
240242

@@ -274,7 +276,7 @@ You can:
274276

275277
{{< partial name="whats-next/whats-next.html" >}}
276278

277-
[1]: https://app.datadoghq.com/llm/settings/evaluations
279+
[1]: https://app.datadoghq.com/llm/evaluations
278280
[2]: /llm_observability/evaluations/managed_evaluations#connect-your-llm-provider-account
279281
[3]: /service_management/events/explorer/facets/
280282
[4]: /monitors/

content/en/llm_observability/evaluations/managed_evaluations/_index.md

Lines changed: 6 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ aliases:
1717

1818
## Overview
1919

20-
Managed evaluations are built-in tools to assess your LLM application on dimensions like quality, security, and safety. By enabling them, you can assess the effectiveness of your application's responses, including detection of negative sentiment, topic relevancy, toxicity, failure to answer and hallucination.
20+
Managed evaluations are built-in tools to assess your LLM application on dimensions like quality, security, and safety. By creating them, you can assess the effectiveness of your application's responses, including detection of sentiment, topic relevancy, toxicity, failure to answer, and hallucination.
2121

2222
LLM Observability associates evaluations with individual spans so you can view the inputs and outputs that led to a specific evaluation.
2323

@@ -98,7 +98,7 @@ If your LLM provider restricts IP addresses, you can obtain the required IP rang
9898

9999
## Create new evaluations
100100

101-
1. Navigate to [**AI Observability > Settings > Evaluations**][2].
101+
1. Navigate to [**AI Observability > Evaluations**][2].
102102
1. Click on the **Create Evaluation** button on the top right corner.
103103
1. Select a specific managed evaluation. This will open the evalution editor window.
104104
1. Select the LLM application(s) you want to configure your evaluation for.
@@ -109,14 +109,12 @@ If your LLM provider restricts IP addresses, you can obtain the required IP rang
109109
- (Optional) Select what percentage of spans you would like this evaluation to run on by configuring the **sampling percentage**. This number must be greater than `0` and less than or equal to `100` (sampling all spans).
110110
1. (Optional) Configure evaluation options by selecting what subcategories should be flagged. Only available on some evaluations.
111111

112-
After you click **Save**, LLM Observability uses the LLM account you connected to power the evaluation you enabled.
112+
After you click **Save and Publish**, LLM Observability uses the LLM account you connected to power the evaluation you enabled. Alternatively, you can **Save as Draft** and edit or enable them later.
113113

114114
## Edit existing evaluations
115115

116-
1. Navigate to [**AI Observability > Settings > Evaluations**][2].
117-
1. Find on the evaluation you want to edit and toggle the **Enabled Applications** button.
118-
1. Select the edit icon to configure the evaluation for an individual LLM application or click on the application name.
119-
1. Evaluations can be disabled by selecting the disable icon for an individual LLM application.
116+
1. Navigate to [**AI Observability > Evaluations**][2].
117+
1. Hover over the evaluation you want to edit and click the **Edit** button.
120118

121119
### Estimated token usage
122120

@@ -335,7 +333,7 @@ This check ensures that sensitive information is handled appropriately and secur
335333

336334
{{< partial name="whats-next/whats-next.html" >}}
337335

338-
[2]: https://app.datadoghq.com/llm/settings/evaluations
336+
[2]: https://app.datadoghq.com/llm/evaluations
339337
[3]: https://app.datadoghq.com/llm/applications
340338
[4]: /security/sensitive_data_scanner/
341339
[5]: https://docs.datadoghq.com/api/latest/ip-ranges/

content/en/llm_observability/instrumentation/sdk.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2079,7 +2079,7 @@ def llm_call():
20792079
return completion
20802080
{{< /code-block >}}
20812081

2082-
[1]: https://app.datadoghq.com/llm/settings/evaluations
2082+
[1]: https://app.datadoghq.com/llm/evaluations
20832083

20842084
{{% /tab %}}
20852085

441 KB
Loading

0 commit comments

Comments
 (0)