Overview

Chroma is an open-source vector database designed for AI applications, supporting storage of embeddings and metadata for efficient similarity search. It can serve as an AI agent's long-term memory system, enabling agents to remember and retrieve relevant information across sessions.

Installation and Configuration

Installation

pip install chromadb chromadb-server

Startup Modes

Embedded Mode (default, for development):

import chromadb
client = chromadb.Client()

Persistent Mode (for production):

import chromadb
client = chromadb.PersistentClient(path="./chroma_data")

Collection Management

Create/Get Collection

collection = client.get_or_create_collection(
    name="knowledge_base",
    metadata={"description": "Knowledge base collection"}
)

Document Operations

Add Documents

collection.add(
    documents=[
        "Python is a high-level programming language",
        "JavaScript is mainly used for web development",
        "Go language is known for concurrency performance"
    ],
    ids=["doc1", "doc2", "doc3"],
    metadatas=[
        {"language": "programming", "level": "beginner"},
        {"language": "web", "level": "beginner"},
        {"language": "system", "level": "intermediate"}
    ]
)

Custom Embedding Function

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

def embed_texts(texts):
    return model.encode(texts).tolist()

collection.add(
    documents=["Document content..."],
    ids=["doc1"],
    embedding_function=embed_texts
)

Similarity Search

Basic Search

results = collection.query(
    query_texts=["What programming language is suitable for beginners"],
    n_results=3
)

print(results["documents"])  # Document content
print(results["distances"])    # Distance scores

Metadata Filtering

results = collection.query(
    query_texts=["Efficient programming language"],
    n_results=5,
    where={"language": "programming"}
)

Agent Memory Integration Example

import chromadb
from langchain_openai import OpenAIEmbeddings

client = chromadb.PersistentClient(path="./agent_memory")
memory_collection = client.get_or_create_collection("agent_memory")
embeddings = OpenAIEmbeddings()

def remember(topic: str, content: str):
    """Store important information to memory"""
    vector = embeddings.embed_query(content)
    memory_collection.add(
        documents=[content],
        embeddings=[vector],
        ids=[f"mem_{topic}"]
    )
    return f"Remembered information about '{topic}'"

def recall(query: str):
    """Retrieve relevant information from memory"""
    query_vector = embeddings.embed_query(query)
    results = memory_collection.query(
        query_embeddings=[query_vector],
        n_results=3
    )
    if not results["documents"][0]:
        return "No relevant information found"
    return "\n".join(results["documents"][0])

Common Questions

Q1: What embedding model does Chroma use by default?

• Default: SentenceTransformer's all-MiniLM-L6-v2
• Automatically downloaded on first use
• Can specify custom model via embedding_function parameter

Q2: How to choose distance function?

• cosine: Cosine similarity, good for directional similarity
• l2: Euclidean distance, good for numerical magnitude
• ip: Inner product, good for non-normalized vectors

Q3: How to handle large-scale data?

• Use client-server mode for scalability
• Configure HNSW index parameters for better performance
• Regularly clean up expired data

References

• Chroma Official Documentation
• Chroma GitHub
• Sentence Transformers