Conversation Memory

Conversation Memory

Persistent memory systems for LLM conversations including short-term, long-term, and entity-based memory

Capabilities

• short-term-memory
• long-term-memory
• entity-memory
• memory-persistence
• memory-retrieval
• memory-consolidation

Prerequisites

• Knowledge: LLM conversation patterns, Database basics, Key-value stores
• Skills_recommended: context-window-management, rag-implementation

Scope

• Does_not_cover: Knowledge graph construction, Semantic search implementation, Database administration
• Boundaries: Focus is memory patterns for LLMs, Covers storage and retrieval strategies

Ecosystem

Primary_tools

• Mem0 - Memory layer for AI applications
• LangChain Memory - Memory utilities in LangChain
• Redis - In-memory data store for session memory

Patterns

Tiered Memory System

Different memory tiers for different purposes

When to use: Building any conversational AI

interface MemorySystem {
    // Buffer: Current conversation (in context)
    buffer: ConversationBuffer;
    // Short-term: Recent interactions (session)
    shortTerm: ShortTermMemory;
    // Long-term: Persistent across sessions
    longTerm: LongTermMemory;
    // Entity: Facts about people, places, things
    entity: EntityMemory;
}

class TieredMemory implements MemorySystem {
    async addMessage(message: Message): Promise<void> {
        // Always add to buffer
        this.buffer.add(message);
        // Extract entities
        const entities = await extractEntities(message);
        for (const entity of entities) {
            await this.entity.upsert(entity);
        }
        // Check for memorable content
        if (await isMemoryWorthy(message)) {
            await this.shortTerm.add({
                content: message.content,
                timestamp: Date.now(),
                importance: await scoreImportance(message)
            });
        }
    }

    async consolidate(): Promise<void> {
        // Move important short-term to long-term
        const memories = await this.shortTerm.getOld(24 * 60 * 60 * 1000);
        for (const memory of memories) {
            if (memory.importance > 0.7 || memory.referenced > 2) {
                await this.longTerm.add(memory);
            }
            await this.shortTerm.remove(memory.id);
        }
    }

    async buildContext(query: string): Promise<string> {
        const parts: string[] = [];
        // Relevant long-term memories
        const longTermRelevant = await this.longTerm.search(query, 3);
        if (longTermRelevant.length) {
            parts.push('## Relevant Memories\n' +
                longTermRelevant.map(m => `- ${m.content}`).join('\n'));
        }
        // Relevant entities
        const entities = await this.entity.getRelevant(query);
        if (entities.length) {
            parts.push('## Known Entities\n' +
                entities.map(e => `- ${e.name}: ${e.facts.join(', ')}`).join('\n'));
        }
        // Recent conversation
        const recent = this.buffer.getRecent(10);
        parts.push('## Recent Conversation\n' + formatMessages(recent));

        return parts.join('\n\n');
    }
}

Entity Memory

Store and update facts about entities

When to use: Need to remember details about people, places, things

interface Entity {
    id: string;
    name: string;
    type: 'person' | 'place' | 'thing' | 'concept';
    facts: Fact[];
    lastMentioned: number;
    mentionCount: number;
}
interface Fact {
    content: string;
    confidence: number;
    source: string;  // Which message this came from
    timestamp: number;
}

class EntityMemory {
    async extractAndStore(message: Message): Promise<void> {
        // Use LLM to extract entities and facts
        const extraction = await llm.complete(`
            Extract entities and facts from this message.
            Return JSON: { "entities": [
                { "name": "...", "type": "...", "facts": ["..."] }
            ]}

            Message: "${message.content}"
        `);
        const { entities } = JSON.parse(extraction);
        for (const entity of entities) {
            await this.upsert(entity, message.id);
        }
    }

    async upsert(entity: ExtractedEntity, sourceId: string): Promise<void> {
        const existing = await this.store.get(entity.name.toLowerCase());
        if (existing) {
            // Merge facts, avoiding duplicates
            for (const fact of entity.facts) {
                if (!this.hasSimilarFact(existing.facts, fact)) {
                    existing.facts.push({
                        content: fact,
                        confidence: 0.9,
                        source: sourceId,
                        timestamp: Date.now()
                    });
                }
            }
            existing.lastMentioned = Date.now();
            existing.mentionCount++;
            await this.store.set(existing.id, existing);
        } else {
            // Create new entity
            await this.store.set(entity.name.toLowerCase(), {
                id: generateId(),
                name: entity.name,
                type: entity.type,
                facts: entity.facts.map(f => ({
                    content: f,
                    confidence: 0.9,
                    source: sourceId,
                    timestamp: Date.now()
                })),
                lastMentioned: Date.now(),
                mentionCount: 1
            });
        }
    }
}

Memory-Aware Prompting

Include relevant memories in prompts

When to use: Making LLM calls with memory context

async function promptWithMemory(
    query: string,
    memory: MemorySystem,
    systemPrompt: string
): Promise<string> {
    // Retrieve relevant memories
    const relevantMemories = await memory.longTerm.search(query, 5);
    const entities = await memory.entity.getRelevant(query);
    const recentContext = memory.buffer.getRecent(5);

    // Build memory-augmented prompt
    const prompt = `
${systemPrompt}

## User Context
${entities.length ? `Known about user:\n${entities.map(e =>
    `- ${e.name}: ${e.facts.map(f => f.content).join('; ')}`
).join('\n')}` : ''}

${relevantMemories.length ? `Relevant past interactions:\n${relevantMemories.map(m =>
    `- [${formatDate(m.timestamp)}] ${m.content}`
).join('\n')}` : ''}

## Recent Conversation
${formatMessages(recentContext)}

## Current Query
${query}
    `.trim();

    const response = await llm.complete(prompt);

    // Extract any new memories from response
    await memory.addMessage({ role: 'assistant', content: response });

    return response;
}

Sharp Edges

Memory store grows unbounded, system slows

Severity: HIGH

Situation: System slows over time, costs increase

Symptoms:

• Slow memory retrieval
• High storage costs
• Increasing latency over time

Why this breaks: Every message stored as memory. No cleanup or consolidation. Retrieval over millions of items.

Recommended fix:

// Implement memory lifecycle management

class ManagedMemory {
    // Limits
    private readonly SHORT_TERM_MAX = 100;
    private readonly LONG_TERM_MAX = 10000;
    private readonly CONSOLIDATION_INTERVAL = 24 * 60 * 60 * 1000;

    async add(memory: Memory): Promise<void> {
        // Score importance before storing
        const score = await this.scoreImportance(memory);
        if (score < 0.3) return;  // Don't store low-importance

        memory.importance = score;
        await this.shortTerm.add(memory);

        // Check limits
        await this.enforceShortTermLimit();
    }

    async enforceShortTermLimit(): Promise<void> {
        const count = await this.shortTerm.count();
        if (count > this.SHORT_TERM_MAX) {
            // Consolidate: move important to long-term, delete rest
            const memories = await this.shortTerm.getAll();
            memories.sort((a, b) => b.importance - a.importance);