-
Notifications
You must be signed in to change notification settings - Fork 51
Open
Description
So each sample in the benchmark contains input(query) and output(reference, expected_steps, order_matters) which is visible in the langsmith UI, but when i try to download the same task benchmarks locally using this code
download_public_dataset(
"https://smith.langchain.com/public/452ccafc-18e1-4314-885b-edd735f17b9d/"
)
with open("./452ccafc-18e1-4314-885b-edd735f17b9d.json", "r", encoding="utf-8") as f:
print(json.dumps(json.load(f)[0], indent=2, sort_keys=True))
Output:
[
{
"created_at": "2024-08-29T18:00:33.682146+00:00",
"dataset_id": "3f29798f-5939-4643-bd99-008ca66b72ed",
"id": "1620401b-f41f-4dcc-a2a4-a9026504b2c9",
"inputs": {
"question": "What are the benefits of using LangChain for building chatbots?"
},
"metadata": {
"dataset_split": [
"base"
],
"source": "synthetic"
},
"modified_at": "2024-08-29T18:00:33.682146+00:00",
"outputs": {
"answer": "LangChain offers several benefits for building chatbots, including modular components for handling various tasks, easy integration with multiple language models, built-in support for memory and state management, and a flexible architecture that allows for customization and scalability."
},
"runs": [],
"source_run_id": null
}
]
It doesn't contain all the output variables like intermediate_steps. Is there any other way to download the complete benchmark dataset for a task locally?
Metadata
Metadata
Assignees
Labels
No labels