LEANN
LEANN is an ultra-lightweight, on-device RAG (Retrieval-Augmented Generation) dedicated vector indexing technology developed by StarTrail-org, led by researchers from the Sky Computing Lab at UC Berkeley. To address the issue that conventional vector databases require more storage space than the original data by permanently storing high-dimensional dense vectors of numerous document chunks on disk, LEANN maintains only the graph structure and recomputes the embeddings in real-time only at the time of retrieval.
LEANN is an ultra-lightweight, on-device RAG (Retrieval-Augmented Generation) dedicated vector indexing technology developed by the StarTrail-org, led by researchers from the Sky Computing Lab at UC Berkeley. To address the issue of conventional vector databases requiring more storage space than the original data by permanently storing high-dimensional dense vectors of numerous document chunks on disk, LEANN adopts an architecture that maintains only the graph structure and performs 'selective recomputation' of the embeddings in real-time only when needed during retrieval.
The core operation is similar to a map app that, instead of storing 3D satellite images of every street in the world on disk, remembers only the framework (graph) of major road networks and renders only the necessary sections in real-time when a user explores a specific route. LEANN applies 'high-degree preserving pruning' to drastically reduce the edges of the graph itself by preserving the key nodes that serve as exploration hubs in the proximity graph, and then compresses and stores it in CSR (Compressed Sparse Row) format, reducing storage space by up to 97%.
This efficiency enables high-performance RAG to be executed on personal hardware, such as laptops in offline environments where privacy is paramount or network connectivity is limited, even with millions of documents. During graph traversal, only the necessary nodes are dynamically batched and recomputed for embeddings on hardware accelerators, allowing for 100% private semantic search without transmitting data to external servers, with minimal disk usage.
💻 System Requirements
0 (Can operate on CPU alone, but NVIDIA GPU 4GB+ or Apple Silicon unified memory is recommended to improve real-time re-computation performance for embeddings)
Varies depending on model and data volume, but index file size is reduced by 97% compared to existing vector databases (tens of MB to hundreds of MB for millions of chunks)
⚡ Installation
4-1. Quick Start
uv tool install leann-core --with leann
4-2. Detailed Installation
Clone and navigate to the GitHub repositorygit clone https://github.com/StarTrail-org/LEANN.git cd LEANN
Create and activate a virtual environment using uvuv venv source .venv/bin/activate
Install the leann package and dependencies (leann[cpu] is recommended for CPU-only environments)uv pip install leann
FAQ
What is LEANN?
LEANN is an ultra-lightweight, on-device RAG (Retrieval-Augmented Generation) dedicated vector indexing technology developed by the StarTrail-org, led by researchers from the Sky Computing Lab at UC Berkeley. To address the issue of conventional vector databases requiring more storage space than the original data by permanently storing high-dimensional dense vectors of numerous document chunks on disk, LEANN adopts an architecture that maintains only the graph structure and performs 'selective recomputation' of the embeddings in real-time only when needed during retrieval. The core operation is similar to a map app that, instead of storing 3D satellite images of every street in the world on disk, remembers only the framework (graph) of major road networks and renders only the necessary sections in real-time when a user explores a specific route. LEANN applies 'high-degree preserving pruning' to drastically reduce the edges of the graph itself by preserving the key nodes that serve as exploration hubs in the proximity graph, and then compresses and stores it in CSR (Compressed Sparse Row) format, reducing storage space by up to 97%. This efficiency enables high-performance RAG to be executed on personal hardware, such as laptops in offline environments where privacy is paramount or network connectivity is limited, even with millions of documents. During graph traversal, only the necessary nodes are dynamically batched and recomputed for embeddings on hardware accelerators, allowing for 100% private semantic search without transmitting data to external servers, with minimal disk usage.
When should I use LEANN?
LEANN is an ultra-lightweight, on-device RAG (Retrieval-Augmented Generation) dedicated vector indexing technology developed by StarTrail-org, led by researchers from the Sky Computing Lab at UC Berkeley. To address the issue that conventional vector databases require more storage space than the original data by permanently storing high-dimensional dense vectors of numerous document chunks on disk, LEANN maintains only the graph structure and recomputes the embeddings in real-time only at the time of retrieval.
📝 Update Notes
No update notes yet.
🧪 Related Code of Life
No related Code of Life posts yet.