astr
06–22–25
a system to remember
I was recently onboarded as an AI engineer at MindMe, an early-stage startup on a mission to reduce workplace burnout. My first task was to design a memory system that would allow our AI features (chat/voice conversational agent) to better understand and support users over time.
The benefit of long-term memory is obvious in the context of a wellness product (relevant follow-up questions, response preferences, behavior patterns, etc.), but in the coming age of agentic AI, working memory is vital for every industry.
My team sent me a link for Mem0: a memory API service (if you're willing to give up control/ability to host your own data), but the open source option was more interesting to me. So here we go! This is how I built a long-term, constantly evolving memory system with Mem0's SDK :)
Mem0 solves a key limitation of most LLM systems (their inability to remember across sessions) by providing a persistent, intelligent memory layer that captures user context, preferences, and insights over time. It uses a hybrid architecture (graph, vector, key-value) to store different memory types efficiently and supports integration with OpenAI and LangGraph. Also, it allows us to keep our data secure with best-practices within HIPAA compliance regulation. My focus was on maximizing three core features: semantic storage of user messages, automatic wellness insight extraction, and dynamic user profiling.
Our system works like this:
We use PostgreSQL and pgvector for semantic search (Mem0 automatically initializes a 'mem' table in your DB):
We deliberately keep the storage pattern simple; only manually storing user messages and letting Mem0 handle everything else automatically:
We then retrieve relevant context using semantic search:
And finally, enhance the system prompt with the retrieved context:
The goal is to:
To me, the most interesting part wasn't just using Mem0 out of the box, it was customizing it for our specific use case. I created two custom prompts that focus on wellness-relevant information. One is a custom fact-extraction prompt that tells Mem0 what information to look for when creating memories from a user message:
The other custom prompt tells Mem0 how to update the user's profile:
The custom prompts are automatically triggered during Mem0's internal processing:
The flow is completely automatic:
Looking ahead, the memory system will evolve toward proactive intelligence: automating tasks like check-ins, goal tracking, and personalized recommendations based on user patterns. Future upgrades include intelligent memory consolidation, graph-based reasoning, and temporal analysis. Privacy will remain a priority, with user-controlled retention, differential privacy, and advanced encryption ensuring both intelligence and security grow in tandem.
Memory isn't just a feature, it's the foundation of any agentic project. If we want to build AI tools that actually help people, we have to give them a level of understanding and personalization that goes beyond surface-level interactions.
I'm excited to continue experimenting with Mem0, but equally open to considering alternative approaches. If you have one in mind, let me know! As long as it can deliver:
The benefit of long-term memory is obvious in the context of a wellness product (relevant follow-up questions, response preferences, behavior patterns, etc.), but in the coming age of agentic AI, working memory is vital for every industry.
My team sent me a link for Mem0: a memory API service (if you're willing to give up control/ability to host your own data), but the open source option was more interesting to me. So here we go! This is how I built a long-term, constantly evolving memory system with Mem0's SDK :)
Mem0 solves a key limitation of most LLM systems (their inability to remember across sessions) by providing a persistent, intelligent memory layer that captures user context, preferences, and insights over time. It uses a hybrid architecture (graph, vector, key-value) to store different memory types efficiently and supports integration with OpenAI and LangGraph. Also, it allows us to keep our data secure with best-practices within HIPAA compliance regulation. My focus was on maximizing three core features: semantic storage of user messages, automatic wellness insight extraction, and dynamic user profiling.
Our system works like this:
We use PostgreSQL and pgvector for semantic search (Mem0 automatically initializes a 'mem' table in your DB):
We deliberately keep the storage pattern simple; only manually storing user messages and letting Mem0 handle everything else automatically:
We then retrieve relevant context using semantic search:
And finally, enhance the system prompt with the retrieved context:
The goal is to:
- Automatically extract meaningful insights from conversations
- Build user profiles that deepen over time
- Provide relevant context for future interactions
- Scale without manual intervention
To me, the most interesting part wasn't just using Mem0 out of the box, it was customizing it for our specific use case. I created two custom prompts that focus on wellness-relevant information. One is a custom fact-extraction prompt that tells Mem0 what information to look for when creating memories from a user message:
- People & Relationships: Family, friends, coworkers, therapists, managers
- Problems & Anxieties: Specific fears, triggers, stress patterns
- Work & HR Context: Job challenges, workplace issues, benefits questions
- Wellness & Health: Physical symptoms, mental health patterns, coping strategies
- Goals & Progress: Personal objectives, therapeutic goals, habit tracking
The other custom prompt tells Mem0 how to update the user's profile:
- Problems & Anxieties: Fears such as public speaking or social interactions
- Work Context: Responsibilities of various job roles, place of work
- Patterns: Recurring anxieties such as before presentations or events
The custom prompts are automatically triggered during Mem0's internal processing:
The flow is completely automatic:
- User sends message → "I'm stressed about my presentation tomorrow"
- AI processes with memory context → Retrieves relevant past memories
- AI generates response → "I remember you mentioned presentation anxiety before..."
- Message storage triggers fact extraction → Custom prompt extracts: "User has upcoming presentation", "User experiences presentation stress"
- Profile update → Custom prompt updates user's anxiety patterns and upcoming events
Looking ahead, the memory system will evolve toward proactive intelligence: automating tasks like check-ins, goal tracking, and personalized recommendations based on user patterns. Future upgrades include intelligent memory consolidation, graph-based reasoning, and temporal analysis. Privacy will remain a priority, with user-controlled retention, differential privacy, and advanced encryption ensuring both intelligence and security grow in tandem.
Memory isn't just a feature, it's the foundation of any agentic project. If we want to build AI tools that actually help people, we have to give them a level of understanding and personalization that goes beyond surface-level interactions.
I'm excited to continue experimenting with Mem0, but equally open to considering alternative approaches. If you have one in mind, let me know! As long as it can deliver:
- Genuine personalization based on individual patterns
- Context-aware responses that feel natural
- Progress tracking and goal management
- Trust-building through consistent understanding
- Scalable relationship management