# Architecture (/docs/architecture)
MemexAI has two runtime modes that share the same Postgres schema. The recommended default is the containerized service. Direct Postgres runtime is available for apps that intentionally own database credentials.
The product flow to keep in mind:
```txt
agent / SDK / MCP -> MemexAI service -> Postgres
```
The service path keeps database credentials out of your app, gives every tenant/user the same path isolation rules, and exposes the admin console for inspecting memory files, revisions, access logs, observability, and background dreaming.
## Containerized service mode [#containerized-service-mode]
Use service mode when teams or production apps should not hold database credentials.
```txt
TypeScript app -> @memexai/sdk -> MemexAI service -> Postgres
Python app -> memexai.MemexAI -> MemexAI service -> Postgres
MCP client -> MemexAI service -> Postgres
```
The service handles:
* API key verification.
* Path validation and virtual-to-physical translation.
* SQL reads and writes to `mx_file`, `mx_revision`, and `mx_access_log`.
* The admin UI at `/admin`.
* MCP transport over SSE and stdio.
* BM25-first memory search, with optional pgvector hybrid search when embeddings are configured.
* Optional background memory consolidation.
## Advanced: direct Postgres runtime mode [#advanced-direct-postgres-runtime-mode]
Use direct Postgres mode only when your application should own database access.
```txt
Your JavaScript app -> @memexai/core -> Postgres
Your Python app -> memexai Python SDK -> Postgres
```
There is no HTTP layer, no service auth, and no separate MemexAI container. Your app passes a Postgres URL, calls `migrate()` on startup or during deploy, then executes memory tools in-process.
## Tool layers [#tool-layers]
### Agentic tools [#agentic-tools]
| Tool | What it does |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `memory_memorize` | Feeds raw text to an inner model that decides what durable facts should be written or patched. |
| `memory_search` | Recalls relevant memory with BM25-first search, optional pgvector hybrid candidates, and model-backed answer synthesis when configured. |
### Raw tools [#raw-tools]
| Tool | What it does |
| ------------------- | ---------------------------------------------------- |
| `memory_list` | Lists visible files for a user. |
| `memory_read` | Reads one file by virtual path. |
| `memory_write` | Creates or overwrites a user file. |
| `memory_patch` | Appends under a heading or replaces exact text. |
| `memory_smart_read` | Builds one bounded context block from visible files. |
## Background dreaming [#background-dreaming]
Service mode can run an opt-in background memory consolidation loop. It is invisible to end users during chat: after a user's memory has been quiet for a configured grace period, the service reads that user's `user/` files and asks a consolidation agent to merge duplicate facts, clean up fragmented notes, resolve direct contradictions, and keep long-running memory files readable for the next agent trajectory.
Dream writes use the same `memory_write` and `memory_patch` path as normal tools, so revisions and access logs still work. The actor is `dream-agent`. Dream reads exclude `user/log.md`, `user/dream-log.md`, files ending in `-log.md`, and files ending in `.log`.
Each scheduler tick only selects users who have **non-excluded file writes newer than their last dream**. If a tick finds no qualifying users it logs a skip message to stdout and returns immediately — no LLM calls are made. When the agent runs but determines nothing needs merging, `files_touched` in `mx_dream_run` is `0` and no `dream-log.md` entry is written. The agent only writes to `dream-log.md` when it actually consolidates something, keeping user memory free of no-op noise.
The operator UX is available both via API and the admin Dreams panel:
| Endpoint | Purpose |
| ------------------------------------------ | ------------------------------------------------------------ |
| `GET /v1/admin/dream/config` | Read `dream_*` settings. |
| `PUT /v1/admin/dream/config` | Update cadence, grace period, write budget, and concurrency. |
| `GET /v1/admin/dream/users` | List dream status, pause flags, errors, and run counts. |
| `PUT /v1/admin/dream/users/:userId/paused` | Pause or resume dreaming for one user. |
Enable the scheduler with `MEMEX_DREAM_ENABLED=true`. The database `dream_enabled` key remains the runtime master switch.
See [Background Dreaming](/docs/operations/dreaming) for skip behavior, no-op runs, pause controls, and deployment guidance.
## Tool call flow [#tool-call-flow]
1. The AI model emits a tool call.
2. A framework adapter receives the call.
3. MemexAI validates the tool name, arguments, and context.
4. Virtual paths like `user/profile.md` are translated to physical paths like `users/user_123/profile.md`.
5. Reads and writes execute against Postgres.
6. Writes create revision snapshots, and reads/writes create access log entries.
7. The result returns to the model or application.
Every step after tool execution is shared by the containerized service, MCP, and in-process runtimes.
## Search and storage flow [#search-and-storage-flow]
MemexAI stores the current memory record in `mx_file`, full write snapshots in `mx_revision`, and every read/write event in `mx_access_log`.
Search starts with Postgres full-text search over the `search_vector` column. In hybrid mode, the service also stores embeddings in Postgres with pgvector and merges lexical and semantic candidates. BM25 remains the default; pgvector is an optional enhancement, not a separate vector database requirement.
# MemexAI Docs (/docs)
MemexAI is drop-in, inspectable memory infrastructure for multi-tenant agents. Run the service beside Postgres, give agents scoped memory tools, and use the admin console to inspect the memory record that changes future behavior.
## Start with the service [#start-with-the-service]
## Product mental model [#product-mental-model]
MemexAI is not transcript RAG. Raw conversations can stay in your app, warehouse, or audit store. MemexAI owns the smaller working set your agent should carry forward: preferences, constraints, corrections, project notes, decisions, shared policies, and source-backed updates.
The default production flow is:
```txt
agent / SDK / MCP -> MemexAI service -> Postgres
```
From there:
* **Architecture:** the service owns database access, while SDKs, adapters, REST, and MCP all route through the same tool engine.
* **Memory model:** agents write structured files; search is BM25-first, with optional pgvector hybrid search inside Postgres.
* **Scopes:** `user/` is private per `userId`; `shared/` is global guidance, read-only by default, and writable only when explicitly enabled.
* **Operations:** revisions, access logs, time travel, observability, and hot/cold files make memory inspectable.
* **Optimization:** teams can update memory schema over time, and dreaming can compact and clean user memory after activity goes quiet.
## How the two paths work [#how-the-two-paths-work]
### 1. Recommended: containerized service [#1-recommended-containerized-service]
Run the MemexAI service alongside Postgres. Your app never gets database credentials; it connects to the service over HTTP with the TypeScript or Python SDK, or through MCP over SSE/stdio.
Use this when you want a deployable memory service with API key auth and the admin UI built in.
If you want your coding agent to perform the integration, start with [Coding Agent Onboarding](/docs/quickstart/agent-onboarding). It uses the same service path, then asks the agent to prove memory with a two-turn test.
### 2. Advanced: direct Postgres runtime [#2-advanced-direct-postgres-runtime]
Skip the MemexAI service container only when your JavaScript or Python app should own the Postgres connection directly. Your app imports the MemexAI runtime, passes a Postgres URL, runs migrations, and executes memory tools in-process.
Use this for embedded deployments, local experiments, or environments where sharing database credentials with the app is an intentional tradeoff.
## Two integration paths [#two-integration-paths]
### Agentic tools [#agentic-tools]
Use this for most assistants. The model gets two tools, and your system prompt gets the MemexAI prompt block.
```ts
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const tools = memory.createAgenticToolset()
// memory_memorize, memory_search
```
Pass both `system` and `tools` into your model call. Tools store and retrieve memory; the prompt block is what makes stored memory available to the next answer.
### Raw tools [#raw-tools]
Use this when your agent or application should manage memory files directly.
```ts
const tools = memory.createRawToolset()
// memory_list, memory_read, memory_write, memory_patch, memory_smart_read
```
## Framework adapters [#framework-adapters]
Drop Memex into the framework you already use.
## Shared memory can guide behavior [#shared-memory-can-guide-behavior]
User memory stores per-user facts, preferences, and project state. Shared memory stores global guidance that every agent can read, such as tool rules, product policies, escalation criteria, and evaluation rubrics.
## Operate and evaluate memory [#operate-and-evaluate-memory]
Benchmarks are useful, but production memory also needs inspectability. Evaluate whether important facts were written, retrieved, cited, and answerable; then check whether an operator can explain failures from revisions, access logs, and time-travel snapshots.
## What MemexAI stores [#what-memexai-stores]
Memory lives in Postgres tables:
| Table | Purpose |
| --------------- | ------------------------------------- |
| `mx_file` | Current memory file contents |
| `mx_revision` | Full write snapshots for auditability |
| `mx_access_log` | Lightweight read/write activity |
| `mx_migration` | Applied schema migrations |
Agents use virtual paths like `user/profile.md` and `shared/policy.md`. MemexAI translates those paths to physical database paths and enforces user isolation.
## Community / Support [#community--support]
Got a question, found a bug, or want to share how you're using MemexAI? [Join us on Slack →](https://join.slack.com/t/memexaispace/shared_invite/zt-3yy24alf6-t1wRQsErf09JViHww_qlGw)
# MCP Clients (/docs/mcp)
The MemexAI service exposes the same core tool engine over Model Context Protocol. REST and MCP share path validation, tool execution, revisions, and access logs.
## SSE transport [#sse-transport]
```txt
http://localhost:8080/v1/mcp/sse?userId=user_123&actor=claude&apiKey=dev-agent-key
```
`userId` defaults to `default`, and `actor` defaults to `mcp-client`.
The API key can be sent as:
* `Authorization: Bearer ...`
* `apiKey` query parameter
* `token` query parameter
Query auth exists for MCP clients that cannot set headers on SSE GET requests.
Messages for SSE sessions are posted back to:
```txt
http://localhost:8080/v1/mcp/messages?connectionId=...
```
## Stdio transport [#stdio-transport]
Build the service first:
```bash
bun run build:service
```
Then run the compiled service in stdio mode:
```bash
DATABASE_URL=postgresql://memexai:memexai@localhost:5433/memexai \
MEMEX_API_KEY=dev-agent-key \
node apps/service/dist/index.js --stdio --user-id user_123 --actor claude-desktop
```
## Exposed tools [#exposed-tools]
MCP clients can call the same memory tools as the REST and direct-mode APIs:
* `memory_memorize`
* `memory_search`
* `memory_list`
* `memory_read`
* `memory_write`
* `memory_patch`
* `memory_smart_read`
# Anthropic SDK (/docs/adapters/anthropic)
The Anthropic SDK adapter is available in `@memexai/core` for direct Postgres mode. It converts MemexAI tool definitions into the format the Anthropic API expects and provides a handler for executing tool use blocks.
## Before you start [#before-you-start]
* Use this page when your app runs [direct Postgres mode](/docs/architecture#advanced-direct-postgres-runtime-mode).
* Read [How MemexAI works](/docs/concepts/how-it-works) for the tool-call-to-Postgres flow.
* Read [Prompt block](/docs/concepts/prompt-block) to understand why memory must be included in the system prompt.
* Read [Memory tools](/docs/concepts/memory-tools) to choose agentic tools or raw file tools.
* Read [Memory scopes](/docs/concepts/scopes) before writing paths like `user/profile.md`.
## Install [#install]
```bash
npm install @memexai/core @anthropic-ai/sdk
```
## Usage [#usage]
```ts
import Anthropic from '@anthropic-ai/sdk'
import { createMemex } from '@memexai/core'
import { createAnthropicTools, handleAnthropicToolCall } from '@memexai/core/adapters/anthropic'
const anthropic = new Anthropic()
const memex = createMemex({ databaseUrl: process.env.DATABASE_URL! })
await memex.migrate()
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const tools = createAnthropicTools(memory)
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const messages: Anthropic.MessageParam[] = [
{ role: 'user', content: 'Remember that I prefer concise answers.' },
]
while (true) {
const response = await anthropic.messages.create({
model: 'claude-opus-4-7',
max_tokens: 1024,
system,
tools,
messages,
})
if (response.stop_reason === 'end_turn') {
console.log(response.content)
break
}
if (response.stop_reason === 'tool_use') {
const toolResults: Anthropic.ToolResultBlockParam[] = []
for (const block of response.content) {
if (block.type !== 'tool_use') continue
const result = await handleAnthropicToolCall(block.name, block.input, memory, undefined, block.id)
toolResults.push({ type: 'tool_result', tool_use_id: block.id, content: JSON.stringify(result) })
}
messages.push({ role: 'assistant', content: response.content })
messages.push({ role: 'user', content: toolResults })
}
}
await memex.end()
```
## API reference [#api-reference]
### `createAnthropicTools(target)` [#createanthropictoolstarget]
Returns an array of `AnthropicTool` objects ready to pass to `anthropic.messages.create({ tools })`.
### `handleAnthropicToolCall(toolName, toolInput, target, ctx?, toolUseId?)` [#handleanthropictoolcalltoolname-toolinput-target-ctx-tooluseid]
Executes a single tool use block and returns the result. Pass the `tool_use` block's `id` as `toolUseId` to link revisions and access logs to the specific call.
The adapter only supplies tools. Use `memory.getSystemPrompt(...)` or `memory.getPromptBlock()` so the stored memory can influence the next response.
# CrewAI (/docs/adapters/crewai)
The CrewAI adapter returns MemexAI tools as `@tool`-decorated async functions compatible with CrewAI agents.
## Before you start [#before-you-start]
* Run the [containerized service](/docs/quickstart/docker-service) or another MemexAI service endpoint.
* Read [How MemexAI works](/docs/concepts/how-it-works) for the tool-call-to-Postgres flow.
* Read [Memory tools](/docs/concepts/memory-tools) to choose two agentic tools or the full raw tool set.
* Read [Memory scopes](/docs/concepts/scopes) before writing paths like `user/profile.md`.
## Install [#install]
```bash
pip install memexai crewai
```
## Usage [#usage]
```python
from memexai import MemexAI
from memexai.adapters.crewai import get_crewai_tools
from crewai import Agent, Task, Crew
memex = MemexAI(url="http://localhost:8080", api_key="dev-agent-key")
user = memex.for_user("user_123", actor="assistant")
tools = get_crewai_tools(user)
assistant = Agent(
role="Personal Assistant",
goal="Maintain durable memory about the user across sessions.",
backstory="You remember what matters and surface it when it's relevant.",
tools=tools,
verbose=True,
)
task = Task(
description="Remember that the user prefers quiet neighborhoods near good schools.",
expected_output="Confirmation that the preference was saved to memory.",
agent=assistant,
)
crew = Crew(agents=[assistant], tasks=[task])
result = crew.kickoff()
print(result)
await memex.close()
```
## Available tools [#available-tools]
`get_crewai_tools` returns seven `@tool`-decorated async functions. Each wraps the corresponding `MemexUser` method:
| Tool | Purpose |
| ------------------- | -------------------------------------------------- |
| `memory_list` | List files visible to the current user |
| `memory_read` | Read a single file by virtual path |
| `memory_write` | Create or overwrite a user file |
| `memory_patch` | Apply targeted updates (append, replace) |
| `memory_smart_read` | Read all memory into a bounded context block |
| `memory_search` | BM25 or hybrid pgvector search over memory |
| `memory_memorize` | Feed raw text and let MemexAI decide what to write |
This example exposes all tools so you can see the full adapter shape. For a minimal production setup, assign only `memory_memorize` and `memory_search` to the agent. Include the full set when a task requires direct file control.
# LangChain (/docs/adapters/langchain)
LangChain adapters are available for both the TypeScript and Python SDKs.
## Before you start [#before-you-start]
* Choose [service mode or direct Postgres mode](/docs/architecture).
* Read [How MemexAI works](/docs/concepts/how-it-works) for the tool-call-to-Postgres flow.
* Read [Prompt block](/docs/concepts/prompt-block) to understand why memory must be included in the system prompt.
* Read [Memory tools](/docs/concepts/memory-tools) to choose agentic tools or raw file tools.
* Read [Memory scopes](/docs/concepts/scopes) before writing paths like `user/profile.md`.
## TypeScript [#typescript]
### Install [#install]
```bash
npm install @memexai/sdk langchain @langchain/core
```
For direct Postgres mode:
```bash
npm install @memexai/core langchain @langchain/core
```
### Service mode [#service-mode]
```ts
import { MemexAI } from '@memexai/sdk'
import { createLangChainTools } from '@memexai/sdk/adapters/langchain'
import { AgentExecutor, createToolCallingAgent } from 'langchain/agents'
import { ChatGoogleGenerativeAI } from '@langchain/google-genai'
import { ChatPromptTemplate } from '@langchain/core/prompts'
const memex = new MemexAI({ url: 'http://localhost:8080', apiKey: process.env.MEMEX_API_KEY! })
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const tools = createLangChainTools(memory)
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const llm = new ChatGoogleGenerativeAI({ model: 'gemini-2.5-flash' })
const prompt = ChatPromptTemplate.fromMessages([
['system', system],
['human', '{input}'],
['placeholder', '{agent_scratchpad}'],
])
const agent = await createToolCallingAgent({ llm, tools, prompt })
const executor = new AgentExecutor({ agent, tools })
const result = await executor.invoke({ input: 'Remember I prefer quiet neighborhoods.' })
console.log(result.output)
```
### Direct Postgres mode [#direct-postgres-mode]
```ts
import { createMemex } from '@memexai/core'
import { createLangChainTools } from '@memexai/core/adapters/langchain'
const memex = createMemex({ databaseUrl: process.env.DATABASE_URL! })
await memex.migrate()
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const tools = createLangChainTools(memory)
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
```
Use `system` in the LangChain prompt template alongside the tools.
***
## Python [#python]
### Install [#install-1]
```bash
pip install memexai langchain langchain-google-genai
```
### Usage [#usage]
```python
from memexai import MemexAI
from memexai.adapters.langchain import get_langchain_tools
from langchain_google_genai import ChatGoogleGenerativeAI
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_core.prompts import ChatPromptTemplate
memex = MemexAI(url="http://localhost:8080", api_key="dev-agent-key")
user = memex.for_user("user_123", actor="assistant")
tools = get_langchain_tools(user)
system = await user.get_system_prompt("You are a helpful assistant with durable user memory.")
llm = ChatGoogleGenerativeAI(model="gemini-2.5-flash")
prompt = ChatPromptTemplate.from_messages([
("system", system),
("human", "{input}"),
("placeholder", "{agent_scratchpad}"),
])
agent = create_tool_calling_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools)
result = await executor.ainvoke({"input": "Remember I prefer quiet neighborhoods."})
print(result["output"])
await memex.close()
```
The Python adapter exposes all seven memory tools: `memory_list`, `memory_read`, `memory_write`, `memory_patch`, `memory_smart_read`, `memory_search`, and `memory_memorize`. For most assistants, start with `memory_memorize` and `memory_search`; see [Memory tools](/docs/concepts/memory-tools) for the tradeoff.
# LlamaIndex (/docs/adapters/llamaindex)
The LlamaIndex adapter wraps all seven MemexAI memory tools as `FunctionTool` objects for use with any LlamaIndex agent.
## Before you start [#before-you-start]
* Run the [containerized service](/docs/quickstart/docker-service) or another MemexAI service endpoint.
* Read [How MemexAI works](/docs/concepts/how-it-works) for the tool-call-to-Postgres flow.
* Read [Memory tools](/docs/concepts/memory-tools) to choose two agentic tools or the full raw tool set.
* Read [Memory scopes](/docs/concepts/scopes) before writing paths like `user/profile.md`.
## Install [#install]
```bash
pip install memexai llama-index-core
```
## Usage [#usage]
```python
from memexai import MemexAI
from memexai.adapters.llamaindex import get_llamaindex_tools
from llama_index.core.agent import FunctionCallingAgentWorker
from llama_index.llms.google import Gemini
memex = MemexAI(url="http://localhost:8080", api_key="dev-agent-key")
user = memex.for_user("user_123", actor="assistant")
tools = get_llamaindex_tools(user)
llm = Gemini(model="models/gemini-2.5-flash")
worker = FunctionCallingAgentWorker.from_tools(tools, llm=llm, verbose=True)
agent = worker.as_agent()
response = await agent.achat("Remember that I prefer quiet neighborhoods near good schools.")
print(response)
await memex.close()
```
## Available tools [#available-tools]
`get_llamaindex_tools` returns seven `FunctionTool` objects:
| Tool | Purpose |
| ------------------- | -------------------------------------------------- |
| `memory_list` | List files visible to the current user |
| `memory_read` | Read a single file by virtual path |
| `memory_write` | Create or overwrite a user file |
| `memory_patch` | Apply targeted updates (append, replace) |
| `memory_smart_read` | Read all memory into a bounded context block |
| `memory_search` | BM25 or hybrid pgvector search over memory |
| `memory_memorize` | Feed raw text and let MemexAI decide what to write |
This example exposes all tools so you can see the full adapter shape. For most assistants, passing only `memory_memorize` and `memory_search` is enough. Include the full set when the agent needs direct file control.
# OpenAI SDK (/docs/adapters/openai)
The OpenAI adapter is available in `@memexai/sdk` for the service mode. It returns a `definitions` array for the API call and an `execute` method for handling tool call results.
## Before you start [#before-you-start]
* Use this page with the [containerized service](/docs/quickstart/docker-service) and `@memexai/sdk`.
* Read [How MemexAI works](/docs/concepts/how-it-works) for the tool-call-to-Postgres flow.
* Read [Prompt block](/docs/concepts/prompt-block) to understand why memory must be included in the system prompt.
* Read [Memory tools](/docs/concepts/memory-tools) to choose agentic tools or raw file tools.
* Read [Memory scopes](/docs/concepts/scopes) before writing paths like `user/profile.md`.
## Install [#install]
```bash
npm install @memexai/sdk openai
```
## Usage [#usage]
```ts
import OpenAI from 'openai'
import { MemexAI } from '@memexai/sdk'
import { createOpenAITools } from '@memexai/sdk/adapters/openai'
const openai = new OpenAI()
const memex = new MemexAI({ url: 'http://localhost:8080', apiKey: process.env.MEMEX_API_KEY! })
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const { definitions, execute } = createOpenAITools(memory)
const messages: OpenAI.Chat.ChatCompletionMessageParam[] = [
{ role: 'system', content: system },
{ role: 'user', content: 'Remember that I prefer concise answers.' },
]
while (true) {
const response = await openai.chat.completions.create({
model: 'gpt-4.1-mini',
messages,
tools: definitions,
tool_choice: 'auto',
})
const choice = response.choices[0]
messages.push(choice.message)
if (choice.finish_reason === 'stop') {
console.log(choice.message.content)
break
}
if (choice.finish_reason === 'tool_calls' && choice.message.tool_calls) {
for (const toolCall of choice.message.tool_calls) {
const result = await execute({
name: toolCall.function.name,
arguments: toolCall.function.arguments,
toolCallId: toolCall.id,
})
messages.push({
role: 'tool',
tool_call_id: toolCall.id,
content: JSON.stringify(result),
})
}
}
}
```
## API reference [#api-reference]
### `createOpenAITools(memory)` [#createopenaitoolsmemory]
Returns `{ definitions, execute }`.
* `definitions` — array of `{ type: 'function', name, description, parameters }` objects, ready to pass as `tools` in a chat completions request.
* `execute({ name, arguments, toolCallId? })` — executes a tool call and returns the result. The `toolCallId` links the execution to the correct revision and access log entry.
The `arguments` field can be either a JSON string (as the OpenAI API returns it) or a pre-parsed object — `normalizeArguments` handles both cases automatically.
The adapter only supplies tools. Use `memory.getSystemPrompt(...)` or `memory.getPromptBlock()` so the stored memory can influence the next response.
# Vercel AI SDK (/docs/adapters/vercel-ai)
The Vercel AI SDK adapter wires MemexAI memory tools into any `generateText`, `streamText`, or `generateObject` call.
## Before you start [#before-you-start]
* Choose [service mode or direct Postgres mode](/docs/architecture).
* Read [How MemexAI works](/docs/concepts/how-it-works) for the tool-call-to-Postgres flow.
* Read [Memory tools](/docs/concepts/memory-tools) to choose agentic tools or raw file tools.
* Read [Memory scopes](/docs/concepts/scopes) before writing paths like `user/profile.md`.
## Install [#install]
```bash
npm install @memexai/sdk ai
```
For direct Postgres mode:
```bash
npm install @memexai/core ai
```
## Service mode (recommended) [#service-mode-recommended]
Use `@memexai/sdk` when connecting to the [containerized MemexAI service](/docs/quickstart/docker-service).
```ts
import { MemexAI } from '@memexai/sdk'
import { generateText, stepCountIs } from 'ai'
import { createGoogleGenerativeAI } from '@ai-sdk/google'
const memex = new MemexAI({ url: 'http://localhost:8080', apiKey: process.env.MEMEX_API_KEY! })
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const result = await generateText({
model: createGoogleGenerativeAI()('gemini-2.5-flash'),
system,
prompt: 'Remember that I prefer quiet neighborhoods near good schools.',
tools: memory.createAgenticToolset(),
stopWhen: stepCountIs(5),
})
console.log(result.text)
```
The instance method `memory.createAgenticToolset()` returns Vercel AI-compatible tools directly. Pair it with `memory.getSystemPrompt(...)` so stored memory can influence the next response.
## Direct Postgres mode [#direct-postgres-mode]
Use `@memexai/core` when your app owns the Postgres connection.
```ts
import { createMemex } from '@memexai/core'
import { generateText, stepCountIs } from 'ai'
import { createGoogleGenerativeAI } from '@ai-sdk/google'
const google = createGoogleGenerativeAI()
const memex = createMemex({ databaseUrl: process.env.DATABASE_URL!, model: google('gemini-2.5-flash') })
await memex.migrate()
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const result = await generateText({
model: google('gemini-2.5-flash'),
system,
prompt: 'Remember that I prefer quiet neighborhoods near good schools.',
tools: memory.createAgenticToolset(),
stopWhen: stepCountIs(5),
})
await memex.end()
```
## Explicit adapter import [#explicit-adapter-import]
The named export is available if you prefer to import it directly.
```ts
import { createVercelAITools } from '@memexai/sdk/adapters/vercel-ai'
// or for direct mode:
import { createVercelAITools } from '@memexai/core/adapters/vercel-ai'
const agenticTools = createVercelAITools(memory)
const rawTools = createVercelAITools(memory, { mode: 'raw' })
```
## Raw toolset [#raw-toolset]
Pass `{ mode: 'raw' }` to expose the full file-level tool set instead of the two agentic tools.
```ts
tools: memory.createRawToolset()
// memory_list, memory_read, memory_write, memory_patch, memory_smart_read
```
Use raw tools when your agent or workflow should control exactly what gets written.
# Access Logs (/docs/concepts/access-logs)
Every tool call that touches memory creates a row in `mx_access_log`. Unlike revisions, access logs include reads.
## What an access log stores [#what-an-access-log-stores]
| Column | What it stores |
| --------------- | ----------------------------------------------------------- |
| `id` | Unique log ID |
| `file_id` | Foreign key to `mx_file`, nullable |
| `physical_path` | Full path at access time |
| `operation` | `list`, `read`, `write`, `patch`, `search`, or `smart_read` |
| `actor` | Who performed the access |
| `user_id` | The user whose memory was accessed |
| `tool_call_id` | The LLM tool call ID |
| `created_at` | Timestamp |
Access logs do not store file content. Revisions store content for writes.
## Access logs vs observation events [#access-logs-vs-observation-events]
Access logs answer: "What memory did the agent touch?"
Observation events answer: "How did the memory call behave?"
Every core tool execution records a root observation event in `mx_observation_event` with `trace_id`, `span_id`, `duration_ms`, status, tool name, operation, and sanitized attributes. Internal phases such as database reads, BM25 search, hybrid search, embedding, LLM generation, revision writes, and access-log writes are recorded as child spans when they run inside a traced tool call.
Tool responses include `traceId`, `memory_trace_id`, `toolCallId`, and `durationMs`. Search and agentic calls also include best-effort `searchStats` and provider token `usage` when available.
MemexAI keeps this trace data locally because memory-native observability is part of the product: admin tools can join traces back to access logs, revisions, and time-travel snapshots. Optional OpenTelemetry export should mirror sanitized spans for external systems, not replace the local record.
## Why access logs matter [#why-access-logs-matter]
Revision history tells you what changed. Access logs tell you what happened.
With access logs you can reconstruct the agent activity around a session:
1. It listed visible files.
2. It read `user/profile.md`.
3. It searched for relevant memory.
4. It patched `user/profile.md`.
That sequence is not visible from revisions alone.
# Design Principles (/docs/concepts/design-principles)
MemexAI makes a small number of opinionated decisions that affect everything downstream. Understanding them helps you structure memory well and predict behavior when things are working correctly.
## Files, not embeddings [#files-not-embeddings]
Memory is Markdown text stored as rows in Postgres — not embedding vectors in a separate store.
This means memory is human-readable, grep-able, patchable by a script, and auditable with a SQL query. You can open any memory file in the admin UI and understand exactly what the agent knows. You can also run eval fixtures against a known memory state and get reproducible results.
BM25 and memory-link traversal cover most retrieval cases in practice. When you need semantic recall, optional pgvector hybrid search is available via `MEMEX_SEARCH_MODE=auto` — see [Docker service setup](/docs/quickstart/docker-service). Both modes are testable and produce predictable results.
## Path-enforced isolation [#path-enforced-isolation]
Multi-tenancy is enforced in code, not prompts.
When an agent calls a memory tool, MemexAI validates the virtual path and injects the `userId` before touching the database. `user/profile.md` physically becomes `users/{userId}/profile.md`. Agents cannot construct a path that reaches another user's data, regardless of what the model generates.
Shared memory is read-only for agents by default through the same mechanism. A write to `shared/anything.md` is rejected at the path validation layer unless the deployment explicitly enables `MEMEX_SHARED_WRITE_MODE=rw`.
## Deterministic retrieval [#deterministic-retrieval]
`memory_smart_read` returns the same files for the same query on the same data, every time.
Retrieval is BM25 full-text search for initial candidates, followed by forward memory-link traversal within a character budget. Every file in the response carries a `reason` field — `query_match`, `linked`, or `recency` — explaining how it was found.
This makes retrieval testable. You can write an eval that seeds a known memory state and asserts exactly which files are returned and why.
## Memory links keep context connected [#memory-links-keep-context-connected]
Memory links let one durable file point at another without copying the same facts everywhere.
When an agent writes `user/profile.md` containing `[[user/preferences.md]]`, a later smart read that seeds `profile.md` can follow the link to include `preferences.md` in the same response. This keeps hub files and supporting notes connected while preserving an inspectable file model.
Bidirectional backlinks and hub scoring are planned next: notes that point back to a hub file will be able to surface as inbound context, and highly referenced files will be able to rank higher in search.
## Writes are permanently recorded [#writes-are-permanently-recorded]
Every write creates a full content snapshot in `mx_revision`. Nothing is silently overwritten.
Agents can correct memory — a later write can update or contradict an earlier one. But the previous version is always recoverable. The access log records every read and write, including which tool call caused it.
This supports post-hoc debugging: when an agent gives a wrong answer, you can inspect the exact memory state it read and trace every write that produced that state.
## Admin and agent have separate trust levels [#admin-and-agent-have-separate-trust-levels]
Agents and operators are not the same thing. MemexAI enforces this structurally.
* **Agents** authenticate with an API key and can always write to `user/*`. They can read `shared/*`, and can write it only when the deployment enables shared writable mode.
* **Operators** authenticate with an admin secret and can read or write any file, inspect revisions, manage config, and control dreaming behavior.
The boundary is not advisory — it is enforced at the route and path validation level. Changing it requires changing how the service is deployed or extending the path rules in code.
## Vocabulary [#vocabulary]
| Term | Meaning |
| ----------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **User** | The person the AI product serves. Each user has a private `user/` namespace. |
| **Agent** | The LLM acting on behalf of the user. Can read+write `user/*`, read `shared/*`, and write `shared/*` only when shared writable mode is enabled. |
| **Operator** | The team or developer running the deployment. Controls `shared/*`, admin config, and model settings. |
| **Virtual path** | The path agents see: `user/profile.md`, `shared/index.md`. |
| **Physical path** | How paths are stored in Postgres: `users/{userId}/profile.md`. Agents never see these. |
| **Hub file** | A memory file that many others link to or reference. Useful as a stable context anchor. |
| **Memory link** | `[[user/file.md]]` syntax in file content. Creates forward graph edges used by smart\_read traversal. |
| **Scope** | The namespace prefix that determines write access and visibility: `user/` or `shared/`. |
| **Dreaming** | Background consolidation. Merges duplicates, resolves contradictions, and keeps memory clean between sessions. See [Dreaming](/docs/operations/dreaming). |
## Related [#related]
* [Memory scopes](/docs/concepts/scopes)
* [How MemexAI works](/docs/concepts/how-it-works)
* [Shared memory as behavior guide](/docs/concepts/shared-memory)
* [Dreaming](/docs/operations/dreaming)
# How MemexAI Works (/docs/concepts/how-it-works)
MemexAI gives an AI agent a small memory surface it can read and update, plus a prompt block that puts stored memory back into the next model call. The memory is stored as scoped files in Postgres, not as a hidden model state or a vector-only blob.
## The loop [#the-loop]
```txt
conversation happens
-> model calls a memory tool
-> MemexAI validates the request
-> virtual paths are translated to physical paths
-> Postgres stores files, revisions, and access logs
-> the next model call includes the MemexAI prompt block
-> a later response searches or reads memory and changes behavior
```
The agent only sees virtual paths like `user/profile.md` and `shared/policy.md`. MemexAI translates those paths before touching the database and enforces isolation in code.
## What gets stored [#what-gets-stored]
MemexAI stores the durable working set an agent should carry forward:
* user preferences
* profile facts
* commitments
* project notes
* corrections
* decisions
* shared guidance and policies
Raw chat transcripts can still live in your app, warehouse, or audit system. MemexAI is for the smaller memory record the agent should actually use later.
## Prompt block [#prompt-block]
Tools alone are not enough. If the model can write memory but never receives memory guidance on the next call, users still experience a forgetful agent. Use `memory.getSystemPrompt(basePrompt)` or add `await memory.getPromptBlock()` to your system prompt on every agent call.
The prompt block tells the model how to use MemexAI tools and includes shared memory files plus the user's `user/index.md` when present. It is the bridge between stored memory and changed behavior.
## Runtime paths [#runtime-paths]
Most production apps use the containerized service:
```txt
app or framework adapter -> MemexAI service -> Postgres
```
Direct Postgres mode embeds the runtime inside your app:
```txt
app -> @memexai/core or memexai Python runtime -> Postgres
```
Both modes share the same schema, path rules, tools, revisions, and access logs.
## What adapters do [#what-adapters-do]
Adapters do not change how memory works. They only translate MemexAI tools into the shape a framework expects.
For example:
* Vercel AI SDK expects tool objects for `generateText` or `streamText`.
* OpenAI and Anthropic expect tool definitions plus a way to execute tool calls.
* LangChain, LlamaIndex, and CrewAI expect framework-native tool wrappers.
After the adapter receives a tool call, MemexAI still runs the same validation, path translation, SQL, revisions, and access logging.
## Related concepts [#related-concepts]
* [Memory tools](/docs/concepts/memory-tools)
* [Prompt block](/docs/concepts/prompt-block)
* [Memory scopes](/docs/concepts/scopes)
* [Revision history](/docs/concepts/revisions)
* [Access logs](/docs/concepts/access-logs)
* [Architecture](/docs/architecture)
# Memory Tools (/docs/concepts/memory-tools)
MemexAI exposes two tool sets. Most assistants should start with the agentic tools. Use raw tools when the agent or application should control memory files directly.
## Agentic tools [#agentic-tools]
Agentic tools give the model a smaller surface:
| Tool | What it does |
| ----------------- | ------------------------------------------------------------------------------------------------- |
| `memory_memorize` | Takes raw text and decides what durable facts should be written or patched. |
| `memory_search` | Searches memory and returns relevant records, with model-backed answer synthesis when configured. |
Use this mode for assistants, copilots, support agents, and product agents where the model should remember useful facts without managing file paths itself.
```ts
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const tools = memory.createAgenticToolset()
```
Pass both values to the model. `memory_memorize` stores durable facts, `memory_search` retrieves relevant files, and the prompt block tells the model when memory should influence the next response.
In service mode, `memory_memorize` needs an LLM configured on the service with `GEMINI_API_KEY` or `OPENAI_API_KEY`. Without a service model, `memory_search` can still use Postgres full-text search, or hybrid pgvector search when embeddings are configured, but `memory_memorize` cannot decide what to write.
## Raw tools [#raw-tools]
Raw tools expose file-level control:
| Tool | What it does |
| ------------------- | -------------------------------------------------- |
| `memory_list` | Lists visible files. |
| `memory_read` | Reads one file by virtual path. |
| `memory_write` | Creates or overwrites a file under `user/`. |
| `memory_patch` | Appends under a heading or replaces exact text. |
| `memory_smart_read` | Builds a bounded context block from visible files. |
Use raw tools when your workflow has a file plan, needs deterministic writes, or should let the application decide exactly where memory lives.
```ts
const tools = memory.createRawToolset()
```
## Which should an adapter expose? [#which-should-an-adapter-expose]
Adapter examples sometimes show all tools so you can see the full integration shape. For a production assistant, start with two tools:
```txt
memory_memorize
memory_search
```
Add raw tools only when the agent has a reason to manage files directly.
## Tool calls and auditability [#tool-calls-and-auditability]
Every memory access creates an access log. Every write or patch also creates a revision snapshot.
Pass framework tool call IDs through adapter handlers when available. That links model tool calls to the revision and access log rows they caused.
## Related concepts [#related-concepts]
* [How MemexAI works](/docs/concepts/how-it-works)
* [Prompt block](/docs/concepts/prompt-block)
* [Memory scopes](/docs/concepts/scopes)
* [Revision history](/docs/concepts/revisions)
* [Access logs](/docs/concepts/access-logs)
# Prompt Block (/docs/concepts/prompt-block)
Stored memory only matters when it reaches the next model call. MemexAI's prompt block is the bridge between the Postgres-backed memory record and the agent's behavior.
## Use the helper [#use-the-helper]
For most TypeScript integrations, compose your system prompt with `getSystemPrompt`:
```ts
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const result = await generateText({
model,
system: await memory.getSystemPrompt(
'You are a helpful assistant with durable user memory.',
),
prompt: userMessage,
tools: memory.createAgenticToolset(),
stopWhen: stepCountIs(5),
})
```
This keeps the base product prompt and the MemexAI memory instructions together. The same underlying memory section is available through `memory.getPromptBlock()` if your framework needs manual message assembly.
## What it contains [#what-it-contains]
The prompt block tells the model:
* that it has access to MemexAI memory
* when to use `memory_memorize` and `memory_search`
* which virtual paths are writable or read-only
* never to use physical database paths
* shared memory files visible to all users
* the current user's `user/index.md` when present
## Two-turn proof [#two-turn-proof]
Use this as the integration check:
```txt
Turn 1: Remember that I prefer quiet neighborhoods near parks.
Turn 2: What kind of neighborhood do I prefer?
```
A successful integration stores the preference on turn one, includes the MemexAI prompt block on turn two, and answers with the quiet-neighborhood preference without the user repeating it. The admin UI should then show the memory file, revision, and later access log.
## Common failure mode [#common-failure-mode]
Passing tools without the prompt block can create a misleading demo: the agent may store memory, but the next response is still generic because the model was not instructed to retrieve and use it. Treat `tools` plus `system` as the pair that makes memory useful.
## Related concepts [#related-concepts]
* [How MemexAI works](/docs/concepts/how-it-works)
* [Memory tools](/docs/concepts/memory-tools)
* [Shared memory](/docs/concepts/shared-memory)
* [Access logs](/docs/concepts/access-logs)
# Revision History (/docs/concepts/revisions)
Every `memory_write` and `memory_patch` creates a row in `mx_revision`. Reads do not create revisions; reads create access log entries instead.
## What a revision stores [#what-a-revision-stores]
| Column | What it stores |
| --------------- | --------------------------------- |
| `id` | Unique revision ID |
| `file_id` | Foreign key to `mx_file` |
| `physical_path` | Full path at write time |
| `operation` | `write` or `patch` |
| `content_text` | Full file content after the write |
| `reason` | Optional human-readable reason |
| `actor` | Who performed the write |
| `user_id` | The user whose memory changed |
| `tool_call_id` | The LLM tool call ID |
| `created_at` | Timestamp |
## Why snapshots [#why-snapshots]
Memory files are usually small. Full snapshots make point-in-time inspection simple:
* No diff replay chain.
* No corruption risk from broken intermediate patches.
* Straightforward auditing in SQL and the admin UI.
## Reason strings [#reason-strings]
Agents should provide a `reason` when writing or patching memory.
```ts
await memory.writeFile({
path: 'user/profile.md',
content,
reason: 'User confirmed preference for quiet neighborhoods',
})
```
A good reason answers why the write was necessary at that moment.
# Memory Scopes (/docs/concepts/scopes)
MemexAI uses a two-level path namespace. Every path an agent sees is a virtual path. The system translates it to a physical path in the database before SQL runs.
## `user/` [#user]
Virtual paths starting with `user/` belong to the user whose `userId` is in the tool context.
```txt
user/profile.md
user/notes/session-2026-05.md
user/reminders.md
```
When an agent writes `user/profile.md` for `user_123`, MemexAI stores it as:
```txt
users/user_123/profile.md
```
The agent never sees the physical path.
## `shared/` [#shared]
Virtual paths starting with `shared/` are visible to every user. By default they cannot be written by agents.
```txt
shared/index.md
shared/policy.md
shared/faq.md
```
Use shared files for context your agents should always have access to: policies, reference docs, product facts, project canon, style guides, learned procedures, or team guidance.
Operators can opt into collective shared memory:
```bash
MEMEX_SHARED_WRITE_MODE=rw
```
When this is enabled, `memory_write` and `memory_patch` can target `shared/**`, and the prompt block/tool descriptions tell the agent that shared memory is writable. This is useful when a team wants shared project memory to improve over time from real usage: canon, policies, style rules, procedures, and cross-user insights can be patched once and benefit every later user.
Use it only for durable global knowledge. Keep private user facts, secrets, raw transcripts, and untrusted external content out of `shared/**`.
## Blocked paths [#blocked-paths]
MemexAI rejects:
* Physical paths like `users/someone/profile.md`.
* Paths with `..`, `//`, or backslashes.
* Writes to `shared/` when shared writable mode is disabled.
Path isolation is enforced in code, not left to the model prompt.
## Why only two scopes [#why-only-two-scopes]
MemexAI ships with two scopes because most products only need these two trust levels: per-user private and operator-global. Adding more scopes before knowing the shape of real usage would bake in premature assumptions.
Team, workspace, and project scopes are on the roadmap as [named mounts](/docs/roadmap#named-mounts) — register additional memory scopes at init time so a team agent can write to `team/itinerary.md` and `user/prefs.md` in the same call. The current design deliberately avoids those scopes until the pull is real.
## Vocabulary [#vocabulary]
For definitions of user, agent, operator, virtual path, physical path, and other terms used across these docs, see [Design Principles — Vocabulary](/docs/concepts/design-principles#vocabulary).
# Shared Memory as Behavior Guide (/docs/concepts/shared-memory)
MemexAI has two virtual memory scopes:
* `user/` is private to a user and writable by the agent.
* `shared/` is global, read-only for agents by default, and injected into every prompt block automatically.
Shared memory is not just a place to store reusable facts. It is the primary lever for controlling how every agent in your product behaves — and for giving agents knowledge your team maintains, not knowledge extracted from user conversations.
Operators can enable collective shared memory with `MEMEX_SHARED_WRITE_MODE=rw`. In that mode, agent tools can write or patch `shared/**`, and the prompt block/tool descriptions tell agents that shared memory is writable. Use this for project canon, policies, product facts, team workflows, style guides, and learned procedures. Do not store private user facts, secrets, raw transcripts, or untrusted external content in `shared/**`.
## Why collective shared memory resonates [#why-collective-shared-memory-resonates]
Most AI memory starts as personal memory: the agent learns a user's preferences, constraints, and corrections. That is useful, but team products also need a memory surface that compounds across collaborators.
Shared writable mode turns `shared/` into an opt-in collective memory surface. A trusted agent can translate a private user signal into a safe project-level update:
```txt
user/profile.md
Writer A privately wants Mira to fear forgetting her brother.
shared/screenplay-canon.md
Mira's missing brother is an emotional anchor for Act I.
```
Later, another collaborator can add project-wide canon:
```txt
shared/screenplay-canon.md
Leaf color encodes emotional category: blue = grief, silver = joy.
```
When Writer A returns, their agent benefits from the blue/silver leaf canon without reading Writer B's private profile. That is the core pattern: private memory stays scoped, while shared canon, policy, style, and workflow knowledge improve for everyone.
Because shared writes go through the same tool path as user writes, every update still creates a revision and access log. You can inspect who changed the shared brain, why, and through which tool call.
## The always-in-context guarantee [#the-always-in-context-guarantee]
Every prompt block built by `getSystemPrompt()` or `getPromptBlock()` automatically includes:
* `shared/index.md` — if it exists, it is injected verbatim into the system prompt
* Any files referenced from `shared/index.md` can be fetched by the agent via `memory_read`
* `user/index.md` — the current user's memory catalog, also injected automatically
This means anything you write to `shared/index.md` or `shared/claude.md` becomes part of every agent session, for every user, without any extra integration code. It is the closest MemexAI has to a "global instruction" that survives model updates and session boundaries.
```ts
// shared/index.md is injected here automatically
const system = await memory.getSystemPrompt('You are a helpful assistant.')
```
You do not need to pass shared file paths explicitly — the prompt block builder reads them from Postgres and injects them for you.
***
## Use case 1: Internal knowledge base [#use-case-1-internal-knowledge-base]
Shared memory is the right place for facts your team maintains that every agent should know — product information, domain terminology, pricing, FAQs, and any knowledge that changes over time.
Unlike system prompt strings (which require a code deploy to update), shared memory files can be edited in the admin console or via the API and take effect on the next session without touching application code.
**Example: product knowledge base**
```md
# shared/knowledge/products.md
## Plans
- Starter: up to 10k memory operations/month, single user scope
- Pro: unlimited operations, team scopes, priority support
- Enterprise: dedicated instance, SLA, custom retention
## Limits
- Maximum file size: 48 KB per memory file
- Revision retention: configurable, default 90 days
- API rate limit: 200 requests/minute on Starter, 2000 on Pro
## Common questions
**Can I export user memory?** Yes. Admin API GET /v1/admin/files returns all files
for a scope as JSON. You own the data.
**Does dreaming touch shared files?** No. Dream consolidation is scoped to
user/ paths only. Shared files are never modified by background dreaming.
```
**Example: domain vocabulary**
```md
# shared/knowledge/domain.md
## Real estate terminology
- EMI: Equated Monthly Installment — the fixed monthly payment on a home loan
- FSI: Floor Space Index — the ratio of total floor area to plot area
- OC: Occupancy Certificate — issued by the municipality after construction
- RERA: Real Estate Regulatory Authority — regulates project registration in India
- Super built-up area: includes the apartment area plus proportional common area share
- Carpet area: the actual usable floor area within the walls
```
Agents that have this in their prompt block will use these terms correctly and can explain them to users without hallucinating definitions.
***
## Use case 2: Operations manual [#use-case-2-operations-manual]
`shared/` can hold an operations manual — step-by-step procedures, escalation rules, and workflow instructions that define how agents should handle specific situations. Think of it as a runbook that the agent reads before every session.
This is particularly valuable for support agents, onboarding agents, and any agent that handles edge cases differently depending on product state.
**Example: support agent operations manual**
```md
# shared/ops/support-playbook.md
## Escalation criteria
Escalate to a human agent if:
- The user mentions a payment dispute or chargeback
- The user asks for a refund on a subscription older than 30 days
- The conversation has been marked unresolved for more than 2 turns
- The user expresses frustration with phrases like "this is unacceptable" or "I want to speak to a manager"
Do NOT escalate for:
- Password resets (use the self-service flow)
- Feature requests (log them in user/feedback.md)
- Billing questions that can be answered from shared/knowledge/products.md
## Tone rules
- Always address the user by their first name if it is in user/profile.md
- Never make promises about feature timelines
- For pricing questions, always link to the pricing page rather than quoting exact figures
## Memory write policy
- Log every escalation in user/support-log.md with the reason and timestamp
- Do not write personal health, financial, or legal inferences to user memory
- Write corrections explicitly: "User corrected city to Mumbai (was Bangalore)"
```
**Example: onboarding agent runbook**
```md
# shared/ops/onboarding.md
## New user flow
1. Check user/profile.md — if empty, this is the first session
2. Ask the user for their primary goal (buying, renting, investing)
3. Ask for their city and neighborhood preferences
4. Write the answers to user/profile.md with the keys: goal, city, neighborhoods[]
5. Introduce the search tools and explain what will be remembered
## Profile quality signals
A high-quality profile has:
- goal (required)
- city (required)
- budget_range (strongly preferred)
- neighborhoods (at least one)
- deal_breakers (if mentioned)
If any required field is missing after 3 turns, prompt the user directly.
## Common gotchas
- Users often mention budget in EMI terms, not total price. Convert and store both.
- "Quiet" often means low traffic, not rural. Ask a follow-up before writing.
- City names can be ambiguous (Bangalore vs Bengaluru). Normalize to Bengaluru Urban.
```
***
## Use case 3: Agent tool policy [#use-case-3-agent-tool-policy]
When shared memory includes tool usage rules, every agent reads them before deciding which tool to call. This lets you tune agent behavior across sessions without changing prompts in code.
```md
# shared/AGENTS.md
## Memory write policy
- Write user preferences only when the user states them directly.
- Do not infer budget, health, legal, or financial constraints without confirmation.
- When a new statement conflicts with old memory, update the memory file and add a correction note.
- Never write raw PII (full names, phone numbers, national IDs) to memory files.
## Tool selection guide
- Use memory_search before making any personalized recommendation.
- Use memory_patch for small changes (adding a field, updating a value).
- Use memory_write only when creating or completely replacing a file.
- Use memory_memorize when the user states a durable preference, constraint, or decision.
## What not to memorize
- Transient statements ("I'm tired today", "I'm in a hurry")
- Questions the user is exploring, not deciding
- Anything the user asks to keep private
- Session artifacts that don't affect future responses
```
***
## Use case 4: Shared guidance + per-user context [#use-case-4-shared-guidance--per-user-context]
The most powerful patterns combine shared guidance with user-specific memory. The agent gets the rules from `shared/` and the user's current context from `user/` — both injected by the same prompt block.
```txt
shared/AGENTS.md → how the agent should behave (global)
shared/knowledge/*.md → what the agent should know (global)
shared/ops/playbook.md → how to handle situations (global)
user/profile.md → who this user is (per-user)
user/projects.md → what the user is working on (per-user)
user/index.md → catalog of this user's memory (per-user, auto-injected)
```
The agent does not need to fetch shared files explicitly — they are in context. If the shared files are large or numerous, use `shared/index.md` as a routing catalog that tells the agent which files to read for each type of task.
**Example: shared/index.md as a routing catalog**
```md
# MemexAI Shared Memory Index
This file is always in context. Reference it to find the right file to read.
## Knowledge
- Products and pricing → shared/knowledge/products.md
- Domain vocabulary → shared/knowledge/domain.md
- Legal and compliance notes → shared/knowledge/compliance.md
## Operations
- Support escalation playbook → shared/ops/support-playbook.md
- Onboarding agent runbook → shared/ops/onboarding.md
## Agent policies
- Memory write and tool selection policy → shared/AGENTS.md
```
***
## Pair shared and user memory [#pair-shared-and-user-memory]
Shared memory should describe how the agent behaves and what it knows. User memory should describe what is true for a specific user.
```txt
shared/AGENTS.md durable behavior and tool rules
shared/policy.md product or workspace policy
shared/index.md routing catalog — always in context
user/profile.md stable user preferences
user/projects.md active work and constraints
user/index.md catalog of user memory — always in context
```
***
## Evaluate memory, not only prompts [#evaluate-memory-not-only-prompts]
When shared memory guides behavior, include it in eval fixtures. A prompt-only eval can pass while production fails because the real agent also reads shared policy, user preferences, and past corrections.
For each eval case, capture:
| Surface | What to check |
| ------------- | -------------------------------------------------------------------------- |
| Prompt | The system and developer instructions sent to the model |
| Shared memory | Tool rules, policies, and behavioral guidance included in the prompt block |
| User memory | Preferences, constraints, and project facts read for the task |
| Access logs | Which memory files were read during the run |
| Revisions | What changed after the run and why |
The best regression tests check both the response and the memory transition. A correct answer with an incorrect memory write should fail, because it will make a later session worse.
***
## Operational guidance [#operational-guidance]
* Keep `shared/index.md` short — it is injected into every prompt block. Treat it as a routing catalog, not a data store.
* Split large knowledge bases into named files under `shared/knowledge/` and reference them from the index.
* Use headings that make agent retrieval easy — the agent will `memory_read` a file when it needs detail.
* Prefer explicit rules over vague style advice in `shared/AGENTS.md`.
* Keep sensitive, per-user facts out of `shared/`.
* If `MEMEX_SHARED_WRITE_MODE=rw` is enabled, prefer `memory_patch` over full rewrites and review shared revisions regularly.
* Review revisions after changing shared guidance — check the access logs to see if agents are reading the new content.
* Link shared guidance changes to product or eval failures when possible, so the correction trail is clear.
Shared memory is powerful because it persists and affects every user session. Treat it with the same care as prompt templates, policy code, and product configuration.
# Admin Console (/docs/operations/admin-console)
The MemexAI service includes an admin console at `/admin`. Use it to make agent memory visible: open the durable memory files, inspect revision history, review access logs, test memory tools, and monitor memory health.
## Access [#access]
In the Docker service, open:
```txt
http://localhost:8080/admin
```
The admin console uses:
* `MEMEX_ADMIN_SECRET` for admin routes
* `MEMEX_API_KEY` for the Tool Playground
Keep both values out of untrusted browsers. If you expose the console outside a local or private network, put it behind your own authentication and proxy admin API calls server-side. MemexAI also does not authenticate end users; your backend must derive `userId` from trusted server-side identity.
## Files [#files]
The Files view is the fastest way to prove that memory exists.
What you can do:
* Browse `users/{userId}/...` and `shared/...` memory as a tree
* Open current memory file contents
* Edit a file when memory is wrong or stale
* See file-level reads, writes, revision count, latest actor, and nearby file activity
* Jump from a file to its revisions and activity trail
Agent tools use virtual paths such as `user/profile.md`. The admin console shows physical paths such as `users/user_123/profile.md` so operators can inspect exactly what is stored.
## Revisions [#revisions]
Every write creates a `mx_revision` row.
Use revisions to answer:
* What changed?
* Who or what actor changed it?
* Why was the change made?
* Which memory content existed before a correction?
* Did background dreaming write this, or did a user-facing agent write it?
The Revisions view also includes manual pruning for old revision snapshots. Pruning removes old `mx_revision` rows only; current files and access logs remain until your application deletes or prunes them separately.
## Activity and access logs [#activity-and-access-logs]
Access logs show which files agents touched.
Use activity to debug:
* Whether the agent read the file you expected
* Whether a file is hot, stale, or rarely used
* Which user scope, actor, or tool call touched memory
* Whether a behavior issue is a missing write, missing read, or bad prompt context
## Observability [#observability]
The Observability view is the operator dashboard for memory health.
It includes:
* Tool calls
* Error rate
* p95 latency
* Active scopes
* Read/write ratio
* SLA watch state
* Memory topology
* Memory activity over time
* Tool latency over time
* Top accessed files
* Recent failures and slow calls
* Per-user memory hygiene when scoped to a user
* Sanitized observation event stream
Observation events are local, memory-native traces. Root events track each tool execution; child spans show phases such as DB reads, BM25 or hybrid search, embedding, LLM generation, revision writes, and access-log writes. Use `traceId` or `toolCallId` from a tool response to connect a product incident to the exact files read, revisions written, and timing breakdown.
MemexAI should keep this local record even when you export sanitized spans to OpenTelemetry. External trace tools can show service latency, but the admin console can also reconstruct memory state and explain whether a failure was caused by a missing write, a retrieval miss, stale memory, or answer synthesis.
Use this view when a product question becomes operational: "Which files are being used?", "Which users have noisy memory?", "Are writes failing?", "Did a new tool route get slow?", or "Which memory nodes are becoming hot?"
## Tool Playground [#tool-playground]
The Tool Playground lets you call MemexAI tools from a local or private operator session before wiring a full app integration.
Use it to:
* Run raw tools such as `memory_list`, `memory_read`, `memory_write`, and `memory_patch`
* Run agentic tools such as `memory_memorize` and `memory_search` when a model is configured
* Inspect JSON responses and errors
* Copy TypeScript, Python, and framework-specific snippets
* Verify API keys and service wiring
Without a configured model, `memory_search` can still use Postgres full-text search, or hybrid pgvector search when an embedding adapter is configured, while `memory_memorize` returns `MODEL_NOT_CONFIGURED`.
## Dreams [#dreams]
The Dreams panel operates background memory consolidation.
Use it to:
* Read and update dream cadence, grace period, write budget, and concurrency
* Pause or resume dreaming globally
* Pause or resume dreaming for a specific user
* Inspect per-user dream status, failures, last run time, and files touched
* Review dream-agent behavior before enabling it broadly
Dreaming is optional. Start conservative, inspect dream-agent revisions, and widen only after quality and cost are clear.
## Admin API map [#admin-api-map]
The console is backed by the admin API:
| Endpoint | Purpose |
| -------------------------------- | ------------------------------------------------------ |
| `GET /v1/admin/users` | List user scopes derived from memory files |
| `GET /v1/admin/files` | List memory files |
| `GET /v1/admin/files/*` | Read one memory file |
| `PUT /v1/admin/files/*` | Edit one memory file |
| `GET /v1/admin/revisions` | List write revisions |
| `POST /v1/admin/revisions/prune` | Manually prune old revision snapshots |
| `GET /v1/admin/access-logs` | List access log rows |
| `GET /v1/admin/observability/*` | Read memory health, topology, activity, and event data |
| `GET /v1/admin/dream/*` | Read dreaming config and per-user status |
| `PUT /v1/admin/dream/*` | Update dreaming config or pause state |
For production caveats around retention, concurrency, and security boundaries, see [Production Considerations](/docs/operations/production-caveats) and [User ID Trust Model](/docs/operations/trust-model).
# Correction Workflow (/docs/operations/correction-workflow)
Wrong memory should be fixed at the record layer. MemexAI stores durable memory as inspectable files, so an operator can correct the source that future prompt blocks and memory searches will read.
## 1. Find the memory [#1-find-the-memory]
Open the admin UI at `/admin`, choose the affected user, and inspect the Files view. User-scoped memory appears under physical paths like:
```txt
users/{userId}/profile.md
users/{userId}/preferences.md
users/{userId}/projects/current.md
```
If you know the agent-visible path, translate `user/profile.md` to `users/{userId}/profile.md`.
## 2. Check the trail [#2-check-the-trail]
Before editing, review:
* The current file content
* Recent revisions for that file
* Access logs showing which tool read or wrote it
* The actor and reason fields, when present
This helps separate a model misunderstanding from a stale or incorrect durable record.
## 3. Edit the file [#3-edit-the-file]
Use the admin UI editor or the admin file endpoint:
```http
PUT /v1/admin/files/users/{userId}/profile.md
```
Include the corrected content and a short reason such as `corrected user neighborhood preference after support review`.
Edits create normal `mx_revision` rows, so the previous content remains visible in revision history.
For deletion or privacy requests, do not rely on editing alone. Delete or purge matching rows from `mx_file`, `mx_revision`, and `mx_access_log` according to your retention policy.
## 4. Verify the next prompt block [#4-verify-the-next-prompt-block]
Fetch the prompt block for the user:
```http
GET /v1/prompt-block?userId={userId}
```
Confirm the corrected memory appears and the wrong fact no longer appears in the injected context.
## 5. Run a two-turn check [#5-run-a-two-turn-check]
Ask the agent a question that depends on the corrected memory. Check that the corrected fact is present in context and used by the agent without requiring the user to restate it. Treat failures as prompt or tooling regressions to investigate.
For higher-risk products, keep a small regression script with the exact correction scenario so the team can rerun it after changes to prompts, adapters, or memory tooling.
## When to delete instead of edit [#when-to-delete-instead-of-edit]
Delete or blank a memory file when the fact should no longer be retained at all. Edit when the fact is still useful but wrong, stale, or too vague.
If your app has compliance deletion requirements, handle account-level deletion in your application workflow and include MemexAI's `mx_file`, `mx_revision`, and `mx_access_log` data in that process.
## Dreaming interactions [#dreaming-interactions]
Dreaming writes through normal memory tools and creates normal revisions. If you correct memory while dreaming is enabled, inspect the next dream-agent revision for that user. Pause dreaming for that user during sensitive manual review if needed.
See [Background Dreaming](/docs/operations/dreaming) for pause controls.
# Background Dreaming (/docs/operations/dreaming)
Dreaming is optional background memory consolidation for service-mode deployments. Use it when your agents write durable memory during user sessions and you want MemexAI to clean that memory after the session has gone quiet.
In practice, dreaming solves the first wave of memory health and compaction problems:
* Duplicate facts spread across multiple user files
* Fragmented notes that should become one stable record
* Direct contradictions after a user corrects earlier memory
* Long-running memory files that are becoming hard for humans and agents to scan
* Low-signal notes that make future memory search noisy
* Trajectory continuity for agents that return to a user or project later
Dreaming is not a transcript summarizer. Keep raw logs in your app or warehouse. Dreaming works on the durable `user/` memory files that MemexAI already owns.
## How the loop works [#how-the-loop-works]
When `MEMEX_DREAM_ENABLED=true`, the service starts a background scheduler. On each tick it:
1. Reads dream config from `mx_config`.
2. Checks whether the database `dream_enabled` key is enabled.
3. Finds users with non-excluded `user/` memory writes newer than their last dream run.
4. Skips users whose memory has not been quiet for the configured grace period.
5. Runs a bounded consolidation agent for eligible users.
6. Writes changes through normal memory tools.
7. Updates `mx_dream_run` with status, timestamps, counts, and errors.
Dream writes use the same `memory_write` and `memory_patch` path as normal tool calls. They create `mx_revision` rows and access logs. The actor is `dream-agent`.
## What gets skipped [#what-gets-skipped]
Dreaming is designed to avoid unnecessary model calls and audit noise.
| Situation | What happens |
| ----------------------------------------------- | -------------------------------------------------------------- |
| Global dreaming is off | The scheduler does not run dream jobs. |
| A specific user is paused | That user is skipped until resumed. |
| No qualifying users changed memory | The tick logs a skip message and makes no LLM calls. |
| A user only changed excluded log files | The user is not selected for dreaming. |
| A user's memory changed too recently | The user waits until the grace period passes. |
| The consolidation agent finds nothing to update | `files_touched` is `0` and no `dream-log.md` entry is written. |
Dream reads exclude `user/log.md`, `user/dream-log.md`, files ending in `-log.md`, and files ending in `.log`. This keeps audit trails from feeding back into future consolidation.
## What the agent can change [#what-the-agent-can-change]
The dream agent is meant to make memory more readable, not invent new facts.
Good dream updates:
* Merge repeated preferences into one cleaner note
* Move scattered project decisions under a clearer heading
* Replace an old fact when the user explicitly corrected it later
* Preserve important context while reducing low-signal wording
* Update `user/index.md` when the file catalog is stale
Bad dream updates:
* Inferring sensitive facts the user did not state
* Treating raw conversation logs as durable memory
* Rewriting shared memory
* Creating a fake source trail
* Making broad behavioral changes without a concrete memory conflict
## No-op runs [#no-op-runs]
A dream run can complete without touching files. That is intentional.
When the agent reviews memory and decides there is nothing worth consolidating:
* `mx_dream_run.files_touched` is set to `0`
* no `user/dream-log.md` entry is written
* no memory file revision is created
This keeps user memory free of "I looked and did nothing" clutter. Operators can still see the latest status in the Dreams panel or through the admin API.
## Pausing dreaming [#pausing-dreaming]
You can pause dreaming globally or for a specific user.
Global pause:
```bash
MEMEX_DREAM_ENABLED=false
```
Runtime global pause:
```http
PUT /v1/admin/dream/config
```
Set `dream_enabled` to `false`.
Per-user pause:
```http
PUT /v1/admin/dream/users/:userId/paused
```
Use per-user pause when a user's memory needs manual review, when you are debugging a specific account, or when your product wants a user-level opt-out for background consolidation.
## Configuration [#configuration]
The service reads runtime settings from `mx_config` using `dream_*` keys.
| Setting | Purpose |
| ---------------------------- | ----------------------------------------------------------------------- |
| `dream_enabled` | Runtime master switch. |
| `dream_interval_minutes` | How often the scheduler checks for eligible users. |
| `dream_grace_period_minutes` | Quiet period after the latest qualifying write before dreaming can run. |
| `dream_max_writes` | Maximum writes a single dream run may perform. |
| `dream_concurrency` | How many users may be processed concurrently. |
The environment variable `MEMEX_DREAM_ENABLED=true` starts the scheduler in the service process. The database `dream_enabled` key remains the runtime switch once the service is running.
## Admin API [#admin-api]
All admin dream endpoints require `x-admin-secret: `.
| Endpoint | Purpose |
| ------------------------------------------ | -------------------------------------------------------------------------------- |
| `GET /v1/admin/dream/config` | Read dream config. |
| `PUT /v1/admin/dream/config` | Update cadence, grace period, write budget, concurrency, and enabled state. |
| `GET /v1/admin/dream/users` | List per-user dream status, pause flags, error messages, counts, and timestamps. |
| `PUT /v1/admin/dream/users/:userId/paused` | Pause or resume dreaming for one user. |
The admin UI includes a Dreams panel for the same operator workflow.
## Deployment notes [#deployment-notes]
* Dreaming is service-mode only.
* Direct Postgres mode does not start a background service loop.
* Configure an LLM provider for the service before enabling dreaming.
* Keep write budgets conservative at first.
* Inspect revisions after the first few runs before widening cadence or concurrency.
For the product framing behind dreaming, see [Background Dreaming](/dreaming).
# Migrations (/docs/operations/migrations)
MemexAI manages its schema with a lightweight migration runner. Calling `migrate()` creates and updates the tables it owns.
## How it works [#how-it-works]
```ts
const memex = createMemex(process.env.DATABASE_URL!)
await memex.migrate()
```
`migrate()` is idempotent:
1. It creates `mx_migration` if it does not exist.
2. It checks each known migration ID.
3. It runs unapplied SQL in a transaction.
4. It inserts the applied migration ID.
5. It skips already applied migrations.
## Current tables [#current-tables]
| Table | Purpose |
| --------------- | --------------------------------------------------- |
| `mx_migration` | Tracks applied migrations |
| `mx_file` | Stores current memory file content |
| `mx_revision` | Stores write snapshots |
| `mx_access_log` | Stores lightweight read/write activity |
| `mx_dream_run` | Stores per-user background consolidation state |
| `mx_config` | Stores runtime config, including `dream_*` settings |
## When to call migrate [#when-to-call-migrate]
* Recommended service mode: the service runs migrations on startup.
* Advanced direct Postgres mode: call `migrate()` once during startup or deploy.
It is safe to call on every startup. Once migrations are applied, the checks are fast.
# Production Considerations (/docs/operations/production-caveats)
MemexAI's core loop is working: agents can write durable memory, the prompt block can inject that memory into later turns, and operators can inspect files, revisions, and access logs in Postgres-backed deployments.
Before using MemexAI in production, review these known constraints and decide which ones matter for your app's risk profile.
## Concurrent writes [#concurrent-writes]
`memory_write` currently uses last-write-wins semantics. Any patch flow that rewrites a file should be treated the same at the file level. If two agents write the same memory file at the same time, the later write can overwrite the earlier one without an optimistic-lock conflict.
For many early deployments this is acceptable because memory writes happen inside one user session or one service worker. If your app can run multiple agents against the same `user/` path concurrently, route writes through one worker, use narrower file paths, or add application-level serialization around high-value files.
Optimistic locking is on the post-launch hardening list.
## Revision and access log growth [#revision-and-access-log-growth]
MemexAI keeps a revision snapshot for every write and an access-log row for reads and writes. That audit trail is one of the main reasons to use MemexAI, but it also means storage grows with traffic.
The service includes a manual admin prune endpoint for old revisions:
```http
POST /v1/admin/revisions/prune
```
Send `{ "olderThanDays": 90 }` with your admin secret to delete matching `mx_revision` rows. The admin UI also exposes this from the Revisions view.
This removes old revision snapshots only. Current file contents and access logs remain until your application deletes or prunes them separately. `mx_access_log` does not yet have an automatic retention policy.
Automated retention and retention settings in the admin UI are on the roadmap.
## Search modes [#search-modes]
The built-in fallback search uses Postgres full-text search. It is useful for English-like keyword search and simple durable memory recall, but by itself it is not semantic search and it does not guarantee strong recall across all languages, paraphrases, or domain-specific phrasing.
In service mode, `MEMEX_SEARCH_MODE=auto` enables pgvector-backed hybrid search when a Gemini API key is configured. Hybrid search keeps BM25 results and adds semantic candidates inside Postgres using reciprocal rank fusion. If no embedding adapter is configured, MemexAI falls back to BM25-only search.
If recall quality is critical, configure hybrid search and model-backed `memory_search`, keep memory files clear and explicit, and test your app's real queries.
## Dreaming token cost [#dreaming-token-cost]
Dreaming is optional background consolidation. When enabled, it reads a bounded set of user memory files and asks a model to merge duplicates, resolve direct contradictions, and keep long-running files readable.
The current implementation has per-run caps for files, input characters, and writes, plus cadence and grace-period controls. It does not yet enforce a per-user daily token or character budget across dream cycles.
Start with dreaming disabled or conservative. Dreaming attempts to consolidate duplicates and contradictions; it does not guarantee perfect memory quality. Enable it for a small population first, inspect dream-agent revisions, and widen cadence only after the cost and quality are clear.
## Security boundary [#security-boundary]
MemexAI scopes memory by `userId`, but it does not authenticate your end user. In service mode, your backend supplies the `userId` when it calls MemexAI. Treat that value as trusted server-side identity, not a browser-controlled field.
For details, see [User ID Trust Model](/docs/operations/trust-model).
## Correction workflow [#correction-workflow]
Wrong memory should be treated as product data, not a model mystery. Operators can open a file, edit the durable record, and rely on the normal revision trail to preserve what changed and why.
For details, see [Correction Workflow](/docs/operations/correction-workflow).
## What we're working on [#what-were-working-on]
The public [roadmap](/roadmap) tracks production-hardening work including optimistic locking, automated retention, per-user dreaming budgets, per-file dreaming exclusions, PII hooks, and richer memory provenance.
# User ID Trust Model (/docs/operations/trust-model)
MemexAI isolates private memory by `userId`, but it does not authenticate your end user. Your application is responsible for deciding which user is active and passing the correct server-side identifier to MemexAI.
## The core rule [#the-core-rule]
Never let an untrusted browser or client choose the `userId` directly.
In service mode, call MemexAI from your backend after your app has authenticated the user:
```ts
const session = await requireSession(request)
const memory = memex.forUser({
userId: session.user.id,
actor: "support-assistant",
})
```
The `userId` should come from your session, auth provider, tenant resolver, or job runner. It should not come from a raw request body field such as `{ "userId": "..." }` unless your backend has validated that the caller is allowed to act for that user.
## What MemexAI enforces [#what-memexai-enforces]
MemexAI translates virtual paths into physical paths:
| Virtual path | Physical path | Agent access |
| ------------------ | --------------------------- | --------------------------------------------------------------------- |
| `user/profile.md` | `users/{userId}/profile.md` | read and write |
| `shared/policy.md` | `shared/policy.md` | read-only by default; writable only with `MEMEX_SHARED_WRITE_MODE=rw` |
Agents do not see physical paths. They work with `user/` and `shared/` paths, and MemexAI validates writes before touching the database. In the default mode, agent tools cannot write into `shared/`. In shared writable mode, the same runtime validator allows trusted agent writes to `shared/**`.
## What your app enforces [#what-your-app-enforces]
Your app owns:
* End-user authentication
* Tenant and workspace authorization
* Which backend jobs may act for a user
* Which admin users can view or edit memory
* Whether memory should be deleted for account closure or compliance requests
Use separate MemexAI deployments, separate databases, or your own database controls if tenants require hard isolation. MemexAI's built-in boundary is path scoping by trusted `userId`, not a tenant authorization layer.
## Service API keys and admin secrets [#service-api-keys-and-admin-secrets]
Agent routes require `Authorization: Bearer `. Keep this key server-side.
Admin routes require an admin secret header. The service accepts `x-memex-admin-secret` and `x-admin-secret` for compatibility with docs and curl examples.
Do not expose the admin secret to untrusted browsers. If you serve admin workflows publicly, put them behind your own authentication and proxy admin API calls server-side.
## Shared memory [#shared-memory]
`shared/` memory is global guidance. It is useful for policies, tool instructions, product-level behavior guides, project canon, and team workflow lessons. Because agents can read it for every user, avoid placing user-private or tenant-private facts in `shared/` unless that is intentional.
If you enable `MEMEX_SHARED_WRITE_MODE=rw`, treat `shared/` like a collaborative project brain: use `memory_patch`, review revisions, and keep private user facts in `user/**`.
## Operational checklist [#operational-checklist]
* Derive `userId` from authenticated server-side state.
* Use stable internal IDs, not emails or names, for user memory paths.
* Keep agent API keys and admin secrets outside client bundles.
* Use separate keys per environment or service, rotate them through your secret manager, and log which internal principal performed admin actions before exposing admin workflows.
* Review admin access before exposing `/admin` in a public environment.
* Document your deletion and correction path before storing regulated data.
See [Production Considerations](/docs/operations/production-caveats) for the broader launch-time caveats.
# Coding Agent Onboarding (/docs/quickstart/agent-onboarding)
The fastest way to try MemexAI in an existing AI app is to hand your coding agent one public instruction file:
```text
Setup MemexAI by following https://memexai.space/setup.md
```
That file is written for coding agents. It tells them how to inspect your app, choose the right SDK adapter, start the local MemexAI service, wire tools into your model call, add the MemexAI prompt block, and verify that memory works.
## TLDR [#tldr]
The agent will add MemexAI in the smallest useful loop:
1. run or reuse a local MemexAI service at `http://localhost:8080`
2. choose a stable `userId`
3. add MemexAI tools to the model call
4. add the MemexAI prompt block with `memory.getSystemPrompt(...)`
5. run one turn that stores memory
6. run a later turn that recalls it
Storage alone is not success. The proof is that the later answer changes because memory was injected into the model call.
## What the agent will ask [#what-the-agent-will-ask]
The setup file asks the agent to confirm two decisions before editing code:
| Question | Recommended answer |
| ---------------------------- | ------------------ |
| Agent mode or Raw tool mode? | Agent mode |
| Set up local Docker Compose? | Yes |
Agent mode exposes the higher-level memory tools:
* `memory_memorize`
* `memory_search`
Raw tool mode is for advanced integrations and debugging. It may expose file-level tools such as `memory_list`, `memory_read`, `memory_write`, and `memory_patch`.
## What the agent will detect [#what-the-agent-will-detect]
The agent inspects your app before changing it:
* package manager and runtime
* TypeScript, JavaScript, or Python project shape
* installed agent SDK
* model call entrypoint
* existing environment variables
It looks for model calls such as `generateText`, `streamText`, OpenAI chat completions, Anthropic messages, LangChain agents, LlamaIndex agents, or CrewAI crews. The goal is to keep your current model SDK and add memory at the existing agent boundary.
For Vercel AI SDK projects, it uses `@memexai/sdk` and `@memexai/sdk/adapters/vercel-ai`. The agent binds MemexAI tools to your model call and uses `memory.getSystemPrompt(...)` so the MemexAI prompt block is included in the system prompt.
If your SDK does not have a first-party MemexAI adapter yet, the agent should still use your existing language/model SDK when it supports custom tools. Direct HTTP calls are the last reserve. The normal fallback is SDK-native custom tools backed by MemexAI's HTTP contract: fetch schemas from `/v1/tools`, add the prompt block from `/v1/prompt-block`, and execute model tool calls through `/v1/tools/:toolName/execute`.
## What it changes in your app [#what-it-changes-in-your-app]
Most TypeScript integrations end up with this shape:
```ts
const memex = new MemexAI({
url: process.env.MEMEX_URL ?? 'http://localhost:8080',
apiKey: process.env.MEMEX_API_KEY ?? 'dev-agent-key',
})
const memory = memex.forUser({ userId, actor: 'assistant' })
const system = await memory.getSystemPrompt(baseSystemPrompt)
const result = await generateText({
model,
system,
prompt: userMessage,
tools: memory.createAgenticToolset(),
stopWhen: stepCountIs(5),
})
```
The two important pieces are `tools` and `system`. Tools let the model write or search memory. The system prompt block tells the model when memory exists and how to use it on the next call.
## Schema setup [#schema-setup]
After the service is running, the agent inspects your app's domain and drafts a `shared/index.md` before finishing the SDK wiring.
The agent will read your README, system prompts, or data models to understand:
* What your product helps users do
* What facts should survive across sessions
* What guidance the agent needs in shared memory (policies, tool rules, escalation criteria)
It will propose a schema — the memory file conventions and guidelines it will follow — and ask you to confirm or adjust before writing anything. Once confirmed, it writes the schema to `shared/index.md` via the admin API. This schema is automatically injected into every system prompt via `getSystemPrompt()`.
You can review or edit the schema at any time from the admin UI Files tab.
## Under the hood [#under-the-hood]
The local Docker service runs MemexAI plus Postgres. Your app never needs database credentials; it only calls the service with `MEMEX_API_KEY`.
When the model calls `memory_memorize`, MemexAI decides what durable facts are worth keeping, validates the requested virtual path, translates `user/...` to the current user's physical database path, writes the file, and stores a revision plus an access log. When the next model call uses `memory.getSystemPrompt(...)`, MemexAI injects shared memory and the current user's memory index into the system prompt so the model can search, read, and answer from the durable record.
The admin UI shows the same record as files, revisions, access logs, and tool activity. That is the operating surface for debugging wrong or stale memory.
## Local service flow [#local-service-flow]
For local development, the setup file points the agent at MemexAI service mode:
1. Check `http://localhost:8080/health`.
2. If it is already healthy, reuse it.
3. If not, check `.env` for `GEMINI_API_KEY`.
4. Ask you for the Gemini key or ask you to place it in `.env`.
5. Start Docker Compose.
6. Verify `/health`.
7. Open the admin UI.
The local defaults are:
```text
MEMEX_API_KEY=dev-agent-key
MEMEX_ADMIN_SECRET=dev-admin-secret
GEMINI_MODEL=gemini-2.5-flash
```
## Success criteria [#success-criteria]
The setup is not done when packages are installed. It is done when the agent can run a two-turn test:
```text
Remember that I prefer quiet neighborhoods near parks.
```
Then:
```text
What kind of neighborhood do I prefer?
```
A successful setup stores the durable preference on the first turn and recalls it on the second turn. The model call should use Gemini from `.env`, include MemexAI tools, and include the MemexAI prompt block or equivalent system prompt section. Storage alone is not success; the second answer should change because memory was injected.
If the proof fails, ask the agent to check:
* the app and service use the same `MEMEX_API_KEY`
* both turns use the same `userId`
* the service has `GEMINI_API_KEY` or another model provider when using `memory_memorize`
* the second model call includes `memory.getSystemPrompt(...)`
* the admin UI shows a write revision after the first turn
## Admin follow-up [#admin-follow-up]
After the service is running, open:
```text
http://localhost:8080/admin
```
The admin UI lets you inspect files, revisions, access logs, and user memory. Use it to confirm which memories were written and how the agent used them.
Useful next docs:
* [Containerized Service](/docs/quickstart/docker-service)
* [Memory Tools](/docs/concepts/memory-tools)
* [Shared Memory](/docs/concepts/shared-memory)
* [Scopes](/docs/concepts/scopes)
* [Revisions](/docs/concepts/revisions)
* [Access Logs](/docs/concepts/access-logs)
* [Background Dreaming](/docs/operations/dreaming)
# Advanced: Direct Postgres Runtime (/docs/quickstart/direct-postgres)
Direct Postgres mode runs the memory engine inside your application process. The recommended default is the [containerized service](/docs/quickstart/docker-service); use this path only when your app intentionally owns database credentials.
Both direct runtimes take a Postgres URL, run MemexAI migrations, and execute memory tools directly against the database.
## JavaScript/TypeScript direct runtime [#javascripttypescript-direct-runtime]
```bash
npm install @memexai/core ai @ai-sdk/google
```
```ts
import { createMemex } from '@memexai/core'
import { generateText, stepCountIs } from 'ai'
import { createGoogleGenerativeAI } from '@ai-sdk/google'
const google = createGoogleGenerativeAI()
const memex = createMemex({
databaseUrl: process.env.DATABASE_URL!,
model: google('gemini-2.5-flash'),
})
await memex.migrate()
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const result = await generateText({
model: google('gemini-2.5-flash'),
system,
prompt: 'Remember that I prefer quiet neighborhoods near good schools.',
tools: memory.createAgenticToolset(),
stopWhen: stepCountIs(5),
})
console.log(result.text)
await memex.end()
```
## Python direct runtime [#python-direct-runtime]
```bash
python3 -m pip install -e "sdks/python[test]"
```
```python
from memexai import create_memex
memex = await create_memex({
"databaseUrl": "postgresql://memexai:memexai@localhost:5433/memexai",
})
await memex.migrate()
memory = memex.for_user("user_123", actor="assistant")
await memory.write_file(
"user/profile.md",
"# Profile\n\n- Prefers quiet neighborhoods.",
reason="captured preference",
)
result = await memory.search("quiet neighborhoods")
await memex.close()
```
## Admin UI [#admin-ui]
Inspect direct-mode memory with the local admin CLI:
```bash
npx @memexai/admin --database-url postgresql://...
```
The CLI opens `http://localhost:4040/admin`.
# Containerized Service (/docs/quickstart/docker-service)
The containerized service path is the recommended default. Your application does not need database credentials; it connects to MemexAI over HTTP with the TypeScript SDK, Python SDK, or through MCP over SSE/stdio.
## Fastest path [#fastest-path]
If you want a coding agent to do the integration inside your app, paste this into the agent:
```text
Setup MemexAI by following https://memexai.space/setup.md
```
That public setup file tells the agent to inspect your app, choose the right adapter, start the local service, wire memory tools, add the MemexAI prompt block, and run a two-turn memory proof. See [Coding Agent Onboarding](/docs/quickstart/agent-onboarding) for the human-readable version.
If you want to wire it manually, follow this page:
1. Start the Docker Compose service.
2. Connect your app with the TypeScript SDK, Python SDK, or MCP.
3. Use one stable `userId` per end user.
4. Pass both the MemexAI tools and `memory.getSystemPrompt(...)` into every model call.
5. Verify that one turn writes memory and a later turn recalls it.
## What you are starting [#what-you-are-starting]
TLDR: Docker runs Postgres plus a MemexAI HTTP service. Your app calls the service with `MEMEX_API_KEY`; operators use the admin UI with `MEMEX_ADMIN_SECRET`.
Under the hood:
* the service creates the `mx_*` Postgres tables on startup
* agent routes execute memory tools and write revisions/access logs
* admin routes let you inspect and correct memory files
* `memory.getSystemPrompt(...)` fetches the prompt block that makes stored memory affect the next answer
* optional model configuration powers `memory_memorize`, answer synthesis in `memory_search`, hybrid search embeddings, and background dreaming
## Compose file [#compose-file]
```yaml
services:
postgres:
image: postgres:16-alpine
environment:
POSTGRES_USER: memexai
POSTGRES_PASSWORD: memexai
POSTGRES_DB: memexai
volumes:
- memexai_postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U memexai"]
interval: 5s
timeout: 5s
retries: 5
memexai:
image: soorajshankar/memexai:latest
depends_on:
postgres:
condition: service_healthy
environment:
DATABASE_URL: postgresql://memexai:memexai@postgres:5432/memexai
MEMEX_API_KEY: dev-agent-key
MEMEX_ADMIN_SECRET: dev-admin-secret
GEMINI_API_KEY: ...
MEMEX_DREAM_ENABLED: "false"
# MEMEX_SHARED_WRITE_MODE: rw # optional: let agent tools write shared/**
# MEMEX_TELEMETRY_DISABLED: "true" # optional opt-out
ports:
- "8080:8080"
volumes:
memexai_postgres_data:
```
Start it:
```bash
docker compose up -d
```
Verify it:
```bash
curl -fsS http://localhost:8080/health
```
The API and admin UI are available at `http://localhost:8080`. For local defaults, open:
```text
http://localhost:8080/admin?defaultAdminSecret=1&defaultApiKey=1&onboarding=1
```
If your build does not support the default-login query params, open `http://localhost:8080/admin` and enter:
```text
Admin secret: dev-admin-secret
Agent API key: dev-agent-key
```
## Connect with the TypeScript SDK [#connect-with-the-typescript-sdk]
```bash
npm install @memexai/sdk ai @ai-sdk/google
```
```ts
import { MemexAI } from '@memexai/sdk'
import { generateText, stepCountIs } from 'ai'
import { createGoogleGenerativeAI } from '@ai-sdk/google'
const memex = new MemexAI({
url: 'http://localhost:8080',
apiKey: process.env.MEMEX_API_KEY ?? 'dev-agent-key',
})
const memory = memex.forUser({ userId: 'user_123', actor: 'assistant' })
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const result = await generateText({
model: createGoogleGenerativeAI()('gemini-2.5-flash'),
system,
prompt: 'Remember that I prefer quiet neighborhoods near good schools.',
tools: memory.createAgenticToolset(),
stopWhen: stepCountIs(5),
})
console.log(result.text)
```
`memory.getSystemPrompt(...)` adds the MemexAI prompt block to your base prompt. Without that prompt block or an equivalent system section, the tools can store memory, but the next response may not be conditioned on it.
## Verify the memory loop [#verify-the-memory-loop]
Run two separate calls for the same `userId`.
Turn one:
```text
Remember that I prefer quiet neighborhoods near good schools.
```
Turn two:
```text
What kind of neighborhood do I prefer?
```
A successful setup answers from stored memory on turn two. In the admin UI, the Files tab should show the user memory file, Revisions should show the write, and Access Logs should show the later read or search.
If turn one writes memory but turn two stays generic, check that the second model call uses the same `userId` and includes `await memory.getSystemPrompt(...)`.
## Connect with the Python SDK [#connect-with-the-python-sdk]
```bash
python3 -m pip install memexai
```
```python
from memexai import MemexAI
memex = MemexAI(
url="http://localhost:8080",
api_key="dev-agent-key",
)
memory = memex.for_user("user_123", actor="assistant")
result = await memory.search("What does this user prefer?")
print(result.get("answer") or result["results"])
await memex.close()
```
## Connect with MCP [#connect-with-mcp]
MCP clients can connect to the same service over SSE:
```txt
http://localhost:8080/v1/mcp/sse?userId=user_123&actor=claude&apiKey=dev-agent-key
```
Or through stdio after building the service:
```bash
bun run build:service
DATABASE_URL=postgresql://memexai:memexai@localhost:5433/memexai \
MEMEX_API_KEY=dev-agent-key \
node apps/service/dist/index.js --stdio --user-id user_123 --actor claude-desktop
```
See [MCP Clients](/docs/mcp) for details.
## Model configuration [#model-configuration]
LLM-backed `memory_memorize` and agentic `memory_search` are configured on the service, not in the SDK.
```bash
GEMINI_API_KEY=...
GEMINI_MODEL=gemini-2.5-flash
# or
OPENAI_API_KEY=...
OPENAI_MODEL=gpt-4.1-mini
```
Without a service model, `memory_search` still works through Postgres full-text search, or hybrid pgvector search when embeddings are configured. `memory_memorize` returns `MODEL_NOT_CONFIGURED`.
For the smoothest first run, set `GEMINI_API_KEY` on the service so the agentic `memory_memorize` tool can decide what should become durable memory.
## Shared writable mode [#shared-writable-mode]
By default, agents can read `shared/**` but can write only `user/**`. To let agent tools contribute durable global knowledge, project canon, policies, style guides, or learned procedures to `shared/**`, set:
```bash
MEMEX_SHARED_WRITE_MODE=rw
```
Keep this disabled for untrusted public-user agents. When enabled, prefer `memory_patch` over full rewrites and keep private user facts, secrets, raw transcripts, and untrusted external content out of `shared/**`.
## Search configuration [#search-configuration]
MemexAI uses BM25 Postgres full-text search by default. With `MEMEX_SEARCH_MODE=auto` and a Gemini API key, the service also stores pgvector embeddings for memory files and runs hybrid BM25 + vector search with reciprocal rank fusion.
```bash
MEMEX_SEARCH_MODE=auto
GEMINI_API_KEY=...
```
Set `MEMEX_SEARCH_MODE=bm25` to force full-text search only.
## Anonymous telemetry [#anonymous-telemetry]
MemexAI service telemetry is enabled by default for the OSS Docker image and service process. It sends anonymous product usage events to PostHog so the project can understand whether installs reach first memory, use MCP, enable dreaming, and hit service errors.
Telemetry never sends memory content, prompts, file paths, tool arguments, user IDs, API keys, admin secrets, or database URLs.
Disable service telemetry with:
```bash
MEMEX_TELEMETRY_DISABLED=true
```
To use a different PostHog project or a self-hosted collector, set `MEMEX_TELEMETRY_POSTHOG_KEY` and `MEMEX_TELEMETRY_POSTHOG_HOST`.
## Background dreaming [#background-dreaming]
Dreaming is optional background memory consolidation for long-horizon agents. Use it when user memory keeps growing across sessions and you want MemexAI to clean the durable record between conversations: merge duplicate facts, compact fragmented notes, resolve direct corrections, refresh stale indexes, and keep files readable for the next agent trajectory.
Turn on the scheduler with:
```bash
MEMEX_DREAM_ENABLED=true
```
The database `dream_enabled` config key is the runtime master switch. The admin Dreams panel and endpoints under `/v1/admin/dream/*` let operators read and update dream config, list user dream status, pause all dreaming, and pause or resume specific users.
Dreaming skips users with no qualifying memory changes, waits for the configured grace period after the latest write, excludes log files from consolidation, and avoids writing `user/dream-log.md` when there was nothing to update.
See [Background Dreaming](/docs/operations/dreaming) for the full operating model.
# Python SDK (/docs/sdks/python)
The Python SDK connects to the recommended containerized MemexAI service by default. Direct Postgres mode is still available for advanced deployments where your Python app intentionally owns database credentials.
## Install [#install]
From the repository root:
```bash
python3 -m pip install -e "sdks/python[test]"
```
From `sdks/python`:
```bash
python3 -m pip install -e ".[test]"
```
## Service usage [#service-usage]
```python
from memexai import MemexAI
memex = MemexAI(
url="http://localhost:8080",
api_key="dev-agent-key",
)
memory = memex.for_user("user_123", actor="assistant")
await memory.write_file(
"user/profile.md",
"# Profile\n\n- Prefers quiet neighborhoods.",
reason="captured preference",
)
result = await memory.read_file("user/profile.md")
print(result["content"])
await memex.close()
```
## Helpers [#helpers]
```python
await memory.list_files(prefix="user/")
await memory.read_file("user/profile.md")
await memory.write_file("user/profile.md", "# Profile", reason="initial write")
await memory.patch_file(
"user/profile.md",
"append_lines",
after_heading="# Profile",
lines=["- Likes good tests"],
reason="new preference",
)
await memory.search("quiet neighborhoods")
await memory.memorize("Remember that the user prefers quiet neighborhoods.")
```
## Framework adapters [#framework-adapters]
| Framework | Page |
| ---------- | ----------------------------------------------- |
| LangChain | [LangChain adapter](/docs/adapters/langchain) |
| LlamaIndex | [LlamaIndex adapter](/docs/adapters/llamaindex) |
| CrewAI | [CrewAI adapter](/docs/adapters/crewai) |
## Advanced: direct Postgres [#advanced-direct-postgres]
Use `create_memex` only when your Python app should connect directly to Postgres.
```python
from memexai import create_memex
memex = await create_memex({
"databaseUrl": "postgresql://memexai:memexai@localhost:5433/memexai",
})
await memex.migrate()
memory = memex.for_user("user_123", actor="assistant")
result = await memory.search("quiet neighborhoods")
await memex.close()
```
# TypeScript SDKs (/docs/sdks/typescript)
MemexAI has two TypeScript packages:
* `@memexai/sdk` is the recommended TypeScript service client for the containerized MemexAI service.
* `@memexai/core` is the advanced direct Postgres runtime.
## Service SDK [#service-sdk]
```bash
npm install @memexai/sdk
```
```ts
import { MemexAI } from '@memexai/sdk'
const memex = new MemexAI({
url: 'http://localhost:8080',
apiKey: 'dev-agent-key',
})
const memory = memex.forUser({
userId: 'demo_user',
actor: 'assistant',
})
const result = await memory.search('What does this user prefer?')
console.log(result.answer ?? result.results)
```
## Advanced: direct Postgres core runtime [#advanced-direct-postgres-core-runtime]
```bash
npm install @memexai/core
```
```ts
import { createMemex } from '@memexai/core'
const memex = createMemex({
databaseUrl: process.env.DATABASE_URL!,
})
await memex.migrate()
const memory = memex.forUser({ userId: 'demo_user', actor: 'assistant' })
await memory.write(
'user/profile.md',
'# Profile\n\n- Prefers quiet neighborhoods.',
'captured user preference',
)
await memex.end()
```
## Toolsets [#toolsets]
```ts
const system = await memory.getSystemPrompt('You are a helpful assistant with durable user memory.')
const agenticTools = memory.createAgenticToolset()
const rawTools = memory.createRawToolset()
```
Use `system` in your model call alongside the toolset. For lower-level control, `memory.getPromptBlock()` returns only the MemexAI memory section so you can compose the final system prompt yourself.
## Framework adapters [#framework-adapters]
Named adapter imports are available for specific frameworks:
| Framework | Package | Page |
| ------------- | ------------------------------- | ------------------------------------------------- |
| Vercel AI SDK | `@memexai/sdk`, `@memexai/core` | [Vercel AI SDK adapter](/docs/adapters/vercel-ai) |
| Anthropic SDK | `@memexai/core` | [Anthropic adapter](/docs/adapters/anthropic) |
| LangChain | `@memexai/sdk`, `@memexai/core` | [LangChain adapter](/docs/adapters/langchain) |
| OpenAI SDK | `@memexai/sdk` | [OpenAI adapter](/docs/adapters/openai) |