Skip to content

Commit 6c9c70b

Browse files
author
Nitin Kanukolanu
committed
Updates
1 parent 0bbff6e commit 6c9c70b

File tree

1 file changed

+32
-125
lines changed

1 file changed

+32
-125
lines changed

‎docs/user_guide/09_svs_vamana.ipynb‎

Lines changed: 32 additions & 125 deletions
Original file line numberDiff line numberDiff line change
@@ -7,15 +7,28 @@
77
"source": [
88
"# SVS-VAMANA Vector Search\n",
99
"\n",
10-
"In this notebook, we will explore SVS-VAMANA, a high-performance vector search algorithm that provides fast approximate nearest neighbor search with vector compression capabilities.\n",
10+
"In this notebook, we will explore SVS-VAMANA (Scalable Vector Search with VAMANA graph algorithm), a graph-based vector search algorithm that is optimized to work with compression methods to reduce memory usage. It combines the Vamana graph algorithm with advanced compression techniques (LVQ and LeanVec) and is optimized for Intel hardware.\n",
1111
"\n",
12-
"SVS-VAMANA offers:\n",
12+
"**How it works**\n",
13+
"\n",
14+
"Vamana builds a single-layer proximity graph and prunes edges during construction based on tunable parameters, similar to HNSW but with a simpler structure. The compression methods apply per-vector normalization and scalar quantization, learning parameters directly from the data to enable fast, on-the-fly distance computations with SIMD-optimized layout Vector quantization and compression.\n",
15+
"\n",
16+
"\n",
17+
"**SVS-VAMANA offers:**\n",
1318
"- **Fast approximate nearest neighbor search** using graph-based algorithms\n",
1419
"- **Vector compression** (LVQ, LeanVec) with up to 87.5% memory savings\n",
1520
"- **Dimensionality reduction** (optional, with LeanVec)\n",
1621
"- **Automatic performance optimization** through CompressionAdvisor\n",
1722
"\n",
18-
"## Table of Contents\n",
23+
"**Use SVS-VAMANA when:**\n",
24+
"- Large datasets where memory is expensive\n",
25+
"- Cloud deployments with memory-based pricing\n",
26+
"- When 90-95% recall is acceptable\n",
27+
"- High-dimensional vectors (>1024 dims) with LeanVec compression\n",
28+
"\n",
29+
"\n",
30+
"\n",
31+
"**Table of Contents**\n",
1932
"\n",
2033
"1. [Prerequisites](#Prerequisites)\n",
2134
"2. [Quick Start with CompressionAdvisor](#Quick-Start-with-CompressionAdvisor)\n",
@@ -51,14 +64,7 @@
5164
{
5265
"cell_type": "code",
5366
"execution_count": 1,
54-
"metadata": {
55-
"execution": {
56-
"iopub.execute_input": "2025-10-15T21:32:08.569435Z",
57-
"iopub.status.busy": "2025-10-15T21:32:08.569310Z",
58-
"iopub.status.idle": "2025-10-15T21:32:08.696705Z",
59-
"shell.execute_reply": "2025-10-15T21:32:08.696395Z"
60-
}
61-
},
67+
"metadata": {},
6268
"outputs": [],
6369
"source": [
6470
"# Import necessary modules\n",
@@ -75,14 +81,7 @@
7581
{
7682
"cell_type": "code",
7783
"execution_count": 2,
78-
"metadata": {
79-
"execution": {
80-
"iopub.execute_input": "2025-10-15T21:32:08.697903Z",
81-
"iopub.status.busy": "2025-10-15T21:32:08.697811Z",
82-
"iopub.status.idle": "2025-10-15T21:32:08.699342Z",
83-
"shell.execute_reply": "2025-10-15T21:32:08.699114Z"
84-
}
85-
},
84+
"metadata": {},
8685
"outputs": [],
8786
"source": [
8887
"# Redis connection\n",
@@ -101,14 +100,7 @@
101100
{
102101
"cell_type": "code",
103102
"execution_count": 3,
104-
"metadata": {
105-
"execution": {
106-
"iopub.execute_input": "2025-10-15T21:32:08.700313Z",
107-
"iopub.status.busy": "2025-10-15T21:32:08.700251Z",
108-
"iopub.status.idle": "2025-10-15T21:32:08.702206Z",
109-
"shell.execute_reply": "2025-10-15T21:32:08.702012Z"
110-
}
111-
},
103+
"metadata": {},
112104
"outputs": [
113105
{
114106
"name": "stdout",
@@ -161,14 +153,7 @@
161153
{
162154
"cell_type": "code",
163155
"execution_count": 4,
164-
"metadata": {
165-
"execution": {
166-
"iopub.execute_input": "2025-10-15T21:32:08.717380Z",
167-
"iopub.status.busy": "2025-10-15T21:32:08.717285Z",
168-
"iopub.status.idle": "2025-10-15T21:32:08.723852Z",
169-
"shell.execute_reply": "2025-10-15T21:32:08.723644Z"
170-
}
171-
},
156+
"metadata": {},
172157
"outputs": [
173158
{
174159
"name": "stdout",
@@ -228,27 +213,14 @@
228213
{
229214
"cell_type": "code",
230215
"execution_count": 5,
231-
"metadata": {
232-
"execution": {
233-
"iopub.execute_input": "2025-10-15T21:32:08.724966Z",
234-
"iopub.status.busy": "2025-10-15T21:32:08.724905Z",
235-
"iopub.status.idle": "2025-10-15T21:32:10.740249Z",
236-
"shell.execute_reply": "2025-10-15T21:32:10.739174Z"
237-
}
238-
},
216+
"metadata": {},
239217
"outputs": [
240218
{
241219
"name": "stdout",
242220
"output_type": "stream",
243221
"text": [
244222
"Creating vectors with 512 dimensions (reduced from 1024 if applicable)\n",
245-
"✅ Loaded 10 documents into the index\n"
246-
]
247-
},
248-
{
249-
"name": "stdout",
250-
"output_type": "stream",
251-
"text": [
223+
"✅ Loaded 10 documents into the index\n",
252224
" Index now contains 0 documents\n"
253225
]
254226
}
@@ -319,14 +291,7 @@
319291
{
320292
"cell_type": "code",
321293
"execution_count": 6,
322-
"metadata": {
323-
"execution": {
324-
"iopub.execute_input": "2025-10-15T21:32:10.744612Z",
325-
"iopub.status.busy": "2025-10-15T21:32:10.744310Z",
326-
"iopub.status.idle": "2025-10-15T21:32:10.751707Z",
327-
"shell.execute_reply": "2025-10-15T21:32:10.751275Z"
328-
}
329-
},
294+
"metadata": {},
330295
"outputs": [
331296
{
332297
"name": "stdout",
@@ -377,14 +342,7 @@
377342
{
378343
"cell_type": "code",
379344
"execution_count": 7,
380-
"metadata": {
381-
"execution": {
382-
"iopub.execute_input": "2025-10-15T21:32:10.753759Z",
383-
"iopub.status.busy": "2025-10-15T21:32:10.753565Z",
384-
"iopub.status.idle": "2025-10-15T21:32:10.757377Z",
385-
"shell.execute_reply": "2025-10-15T21:32:10.757018Z"
386-
}
387-
},
345+
"metadata": {},
388346
"outputs": [
389347
{
390348
"name": "stdout",
@@ -463,14 +421,7 @@
463421
{
464422
"cell_type": "code",
465423
"execution_count": 8,
466-
"metadata": {
467-
"execution": {
468-
"iopub.execute_input": "2025-10-15T21:32:10.759026Z",
469-
"iopub.status.busy": "2025-10-15T21:32:10.758895Z",
470-
"iopub.status.idle": "2025-10-15T21:32:10.762424Z",
471-
"shell.execute_reply": "2025-10-15T21:32:10.762083Z"
472-
}
473-
},
424+
"metadata": {},
474425
"outputs": [
475426
{
476427
"name": "stdout",
@@ -521,14 +472,7 @@
521472
{
522473
"cell_type": "code",
523474
"execution_count": 9,
524-
"metadata": {
525-
"execution": {
526-
"iopub.execute_input": "2025-10-15T21:32:10.763978Z",
527-
"iopub.status.busy": "2025-10-15T21:32:10.763840Z",
528-
"iopub.status.idle": "2025-10-15T21:32:10.768306Z",
529-
"shell.execute_reply": "2025-10-15T21:32:10.768005Z"
530-
}
531-
},
475+
"metadata": {},
532476
"outputs": [
533477
{
534478
"name": "stdout",
@@ -574,14 +518,7 @@
574518
{
575519
"cell_type": "code",
576520
"execution_count": 10,
577-
"metadata": {
578-
"execution": {
579-
"iopub.execute_input": "2025-10-15T21:32:10.770248Z",
580-
"iopub.status.busy": "2025-10-15T21:32:10.770114Z",
581-
"iopub.status.idle": "2025-10-15T21:32:10.774772Z",
582-
"shell.execute_reply": "2025-10-15T21:32:10.774498Z"
583-
}
584-
},
521+
"metadata": {},
585522
"outputs": [
586523
{
587524
"name": "stdout",
@@ -591,7 +528,7 @@
591528
"==============================\n",
592529
"Documents: 0\n",
593530
"Vector index size: 0.00 MB\n",
594-
"Total indexing time: 1.36 seconds\n",
531+
"Total indexing time: 1.58 seconds\n",
595532
"Memory efficiency calculation requires documents and vector index size > 0\n"
596533
]
597534
}
@@ -647,14 +584,7 @@
647584
{
648585
"cell_type": "code",
649586
"execution_count": 11,
650-
"metadata": {
651-
"execution": {
652-
"iopub.execute_input": "2025-10-15T21:32:10.776132Z",
653-
"iopub.status.busy": "2025-10-15T21:32:10.776038Z",
654-
"iopub.status.idle": "2025-10-15T21:32:10.779151Z",
655-
"shell.execute_reply": "2025-10-15T21:32:10.778875Z"
656-
}
657-
},
587+
"metadata": {},
658588
"outputs": [
659589
{
660590
"name": "stdout",
@@ -758,28 +688,13 @@
758688
{
759689
"cell_type": "code",
760690
"execution_count": 12,
761-
"metadata": {
762-
"execution": {
763-
"iopub.execute_input": "2025-10-15T21:32:10.780645Z",
764-
"iopub.status.busy": "2025-10-15T21:32:10.780541Z",
765-
"iopub.status.idle": "2025-10-15T21:32:10.783383Z",
766-
"shell.execute_reply": "2025-10-15T21:32:10.783148Z"
767-
}
768-
},
691+
"metadata": {},
769692
"outputs": [
770693
{
771694
"name": "stdout",
772695
"output_type": "stream",
773696
"text": [
774-
"Cleaned up svs_demo index\n",
775-
"\n",
776-
"🎉 SVS-VAMANA demo completed!\n",
777-
"\n",
778-
"Next steps:\n",
779-
"- Try SVS-VAMANA with your own embeddings\n",
780-
"- Experiment with different compression settings\n",
781-
"- Compare performance with FLAT and HNSW algorithms\n",
782-
"- Use hybrid queries for complex search scenarios\n"
697+
"Cleaned up svs_demo index\n"
783698
]
784699
}
785700
],
@@ -789,15 +704,7 @@
789704
" index.delete()\n",
790705
" print(\"Cleaned up svs_demo index\")\n",
791706
"except:\n",
792-
" print(\"- svs_demo index was already deleted or doesn't exist\")\n",
793-
"\n",
794-
"# Note: The manual index wasn't created in this demo, so no need to delete it\n",
795-
"print(\"\\n🎉 SVS-VAMANA demo completed!\")\n",
796-
"print(\"\\nNext steps:\")\n",
797-
"print(\"- Try SVS-VAMANA with your own embeddings\")\n",
798-
"print(\"- Experiment with different compression settings\")\n",
799-
"print(\"- Compare performance with FLAT and HNSW algorithms\")\n",
800-
"print(\"- Use hybrid queries for complex search scenarios\")"
707+
" print(\"- svs_demo index was already deleted or doesn't exist\")"
801708
]
802709
}
803710
],

0 commit comments

Comments
 (0)