Milvus
Leading the global AI infrastructure market with an open-source distributed vector database.
Milvus is currently the most powerful and widely used open-source distributed vector database in the global AI infrastructure market. While traditional relational databases (such as MySQL and PostgreSQL) search by matching text or numerical data with specific keys or queries, Milvus specializes in storing high-dimensional "vector embeddings" generated by AI models and calculating the similarity between them at extremely high speeds. In particular, Milvus has a unique presence in the fields of bioinformatics and chemoinformatics. Core data in the bio field, such as the 3D structure of proteins (e.g., AlphaFold modeling results), the molecular formulas (SMILES) of hundreds of millions of new drug candidates, or large-scale genomic sequence information, is almost impossible to store and compare for similarity using conventional database structures. However, by passing this bio data through deep learning models (such as ESM and Graph Neural Networks), it can be transformed into mathematical vectors ranging from 128 to 512 or even 2048 dimensions. Milvus can safely store these high-dimensional vectors in the billions, and it can process requests such as "Find the base protein that is most similar to this unknown protein structure within 0.01 seconds!" or "Screen other candidate substances with similar binding properties to this compound!" at very high speeds. Furthermore, it is designed with a cloud-native architecture (Kubernetes support) that can effectively handle large amounts of data, allowing computational power and storage capacity to be scaled independently. It also fully supports GPU-accelerated indexing to overcome the limitations of CPU performance. For research teams looking to securely build large-scale drug development pipelines or bio text RAG (Retrieval-Augmented Generation) systems on-premises (on internal servers) or in their own private cloud environments, Milvus is one of the best choices.
💻 System Requirements
Minimum 8GB recommended (as hundreds of millions of vector indices are loaded directly into memory, so the required specifications increase proportionally to the data size)
Supports integration with object storage such as MinIO (default bundled), AWS S3, or Google Cloud Storage
⚡ Installation
### 4-1. Quick Start
This section describes how to quickly set up and run a local vector database using Milvus Lite in a local Jupyter Notebook or a single Python execution environment, allowing you to start development immediately.
```bash
# 1. Install the PyMilvus SDK and necessary tools (Milvus Lite is included)
pip install pymilvus
# 2. Write Python code to connect and use the local DB file immediately
python -c "
from pymilvus import MilvusClient
# The 'milvus_demo.db' file will be created locally and function as a single DB server.
client = MilvusClient('milvus_demo.db')
print('Milvus Lite connection successful and instance initialized!')
"
```
### 4-2. Detailed Installation
This is the standard method for building a standalone Milvus instance in a local environment or on a server using Docker. This approach starts etcd (state management), MinIO (object storage), and the Milvus server in the background simultaneously.
```bash
# 1. Download the official Milvus standalone Docker Compose file (based on the stable version v2.6.16)
wget https://github.com/milvus-io/milvus/releases/download/v2.6.16/milvus-standalone-docker-compose.yml -O docker-compose.yml
# 2. Start the background services using Docker Compose
sudo docker compose up -d
# 3. Check the process status to ensure the containers are running correctly
sudo docker compose ps
# 4. (After confirmation) Use the PyMilvus client to test the functionality
python -c "
from pymilvus import MilvusClient
client = MilvusClient(uri='http://localhost:19530')
print('Successfully connected to the standalone Milvus server!')
"
```🧬 Bio Use Cases
Virtual Screening of Compounds and Drug Discovery
Convert the structures (in SMILES notation) of compound libraries, ranging from tens of millions to billions of compounds, into Morgan Fingerprints (high-density binary vectors with 1024/2048 dimensions) using libraries like RDKit. Load all converted molecular vectors into Milvus, then input the molecular structure of existing drugs that have shown activity against a target protein as a query to derive hundreds of novel drug candidates with the highest similarity based on Tanimoto distance in real-time.
AlphaFold and ESMFold Protein Structure Search
Convert 3D protein structure information into high-dimensional real-valued vectors using large protein language models such as ESM-2 or the geometric embedding extractor (Structural Embedding) of AlphaFold. By indexing the baseline protein structure vectors from around the world in Milvus, you can input a newly identified target protein structure from the lab and quickly trace morphologically/evolutionarily similar protein families within the existing PDB (Protein Data Bank) data and infer their functions.
Building a PubMed-Based Medical/Biotechnology Paper RAG System
Divide decades of PubMed research papers, patent documents, and NCBI gene function description texts into specific semantic paragraphs, and then convert them into embedding vectors using models like BioBERT or ClinicalBERT. Index these in Milvus, and use them as a RAG (Retrieval-Augmented Generation) knowledge base to retrieve relevant information in real-time and feed it into the LLM prompt, enabling a medical large language model (LLM) to provide accurate answers based on the latest clinical knowledge and gene information without hallucinations.
📝 Update Notes
No update notes yet.
🧪 Related Code of Life
No related Code of Life posts yet.