> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/Gentleman-Programming/engram/llms.txt
> Use this file to discover all available pages before exploring further.
# Architecture
> System architecture, components, and data flow in Engram
## System Overview
Engram is built as an **agent-agnostic Go binary** with thin adapter plugins per-agent. The Go binary is the brain — plugins just translate agent-specific events into Engram API calls.
```
┌─────────────────────────────────────────────────────────────┐
│ Agent (OpenCode / Claude Code / Cursor / Windsurf / etc.) │
└─────────────┬───────────────────────────────────────────────┘
│
├─ Plugin (TypeScript adapter) ──────────────────┐
│ • Session tracking │
│ • Auto-start server │
│ • Memory Protocol injection │
│ • Compaction recovery │
└────────────────────────────────────────────────┘
│
├─ MCP stdio (13 memory tools)
├─ HTTP API (port 7437)
│
┌─────────────▼───────────────────────────────────────────────┐
│ Engram Go Binary (single executable) │
│ ┌──────────────────────────────────────────────────────┐ │
│ │ internal/store/ — Core: SQLite + FTS5 + data ops │ │
│ │ internal/server/ — HTTP REST API (port 7437) │ │
│ │ internal/mcp/ — MCP stdio server (13 tools) │ │
│ │ internal/sync/ — Git sync: manifest + chunks │ │
│ │ internal/tui/ — Bubbletea terminal UI │ │
│ └──────────────────────────────────────────────────────┘ │
└─────────────┬───────────────────────────────────────────────┘
│
┌─────────────▼───────────────────────────────────────────────┐
│ SQLite + FTS5 (~/.engram/engram.db) │
│ • sessions │
│ • observations + observations_fts │
│ • user_prompts + prompts_fts │
│ • sync_chunks │
└──────────────────────────────────────────────────────────────┘
```
## Core Components
### 1. Store (`internal/store/store.go`)
The **heart of Engram** — all data operations go through this.
* SQLite connection management (WAL mode, busy timeout, foreign keys)
* Schema creation and migration
* FTS5 index management (synced via triggers)
* Session CRUD (create, end, list)
* Observation CRUD (add, update, delete, soft-delete)
* Full-text search with query sanitization
* Timeline generation (before/after an observation)
* Deduplication (normalized hash + rolling window)
* Topic upserts (same `project + scope + topic_key` updates existing)
* Privacy tag stripping (`` → `[REDACTED]`)
* Export/import (JSON dump of all data)
* Git sync chunk tracking
```go theme={null}
type Store struct {
db *sql.DB
cfg Config
hooks storeHooks
}
type Session struct {
ID string
Project string
Directory string
StartedAt string
EndedAt *string
Summary *string
Status string
}
type Observation struct {
ID int64
SessionID string
Type string
Title string
Content string
Project *string
Scope string
TopicKey *string
RevisionCount int
DuplicateCount int
CreatedAt string
UpdatedAt string
DeletedAt *string
}
```
```go theme={null}
type Config struct {
DataDir string // ~/.engram
MaxObservationLength int // 2000 chars (for truncated results)
MaxContextResults int // 20 observations
MaxSearchResults int // 20 results
DedupeWindow time.Duration // 15 minutes
}
```
The store uses **pure Go SQLite** (`modernc.org/sqlite`) — no CGO, no external dependencies. This enables true cross-platform single binary distribution.
### 2. HTTP Server (`internal/server/server.go`)
REST API on port 7437 for plugins and integrations.
#### Endpoints
* `GET /health` — Health check (`{"status": "ok", "service": "engram", "version": "0.1.0"}`)
* `GET /stats` — Memory statistics (sessions, observations, prompts, projects)
* `POST /sessions` — Create session (`{id, project, directory}`)
* `POST /sessions/{id}/end` — End session (`{summary}`)
* `GET /sessions/recent` — Recent sessions (`?project=X&limit=N`)
* `POST /observations` — Add observation
* `GET /observations/recent` — Recent observations (`?project=X&scope=project|personal&limit=N`)
* `GET /observations/{id}` — Get single observation
* `PATCH /observations/{id}` — Update fields (partial updates)
* `DELETE /observations/{id}` — Delete (`?hard=true` for permanent)
* `GET /search` — FTS5 search (`?q=QUERY&type=TYPE&project=PROJECT&scope=SCOPE&limit=N`)
* `GET /timeline` — Chronological context (`?observation_id=N&before=5&after=5`)
* `POST /prompts` — Save user prompt
* `GET /prompts/recent` — Recent prompts
* `GET /prompts/search` — Search prompts
* `GET /context` — Formatted context from previous sessions (`?project=X&scope=project|personal`)
* `GET /export` — Export all data as JSON
* `POST /import` — Import data from JSON
The HTTP server listens on `127.0.0.1:7437` (localhost only). Override with `ENGRAM_PORT` environment variable.
### 3. MCP Server (`internal/mcp/mcp.go`)
Model Context Protocol server (stdio transport) that exposes 13 memory tools to any MCP-compatible agent.
#### Tool Profiles
Engram supports tool profiles to reduce context usage:
```bash All tools (default) theme={null}
engram mcp
engram mcp --tools=all
```
```bash Agent profile (11 tools agents actually use) theme={null}
engram mcp --tools=agent
# Includes: mem_save, mem_search, mem_context, mem_session_summary,
# mem_session_start, mem_session_end, mem_get_observation,
# mem_suggest_topic_key, mem_capture_passive, mem_save_prompt, mem_update
```
```bash Admin profile (3 tools for TUI/dashboards) theme={null}
engram mcp --tools=admin
# Includes: mem_delete, mem_stats, mem_timeline
```
```bash Custom combination theme={null}
engram mcp --tools=agent,admin
engram mcp --tools=mem_save,mem_search,mem_context
```
The `agent` profile is recommended for production use — it excludes rarely-used admin tools, saving \~800 tokens per context window.
#### Server Instructions
The MCP server includes instructions for Claude Code's Tool Search:
```
Engram provides persistent memory that survives across sessions and context
compactions. Search these tools when you need to: save decisions, bugs,
architecture choices, or discoveries to memory; recall or search past work
from previous sessions; manage coding session lifecycle (start, end,
summarize); recover context after compaction. Key tools: mem_save,
mem_search, mem_context, mem_session_summary, mem_get_observation,
mem_suggest_topic_key.
```
### 4. Sync (`internal/sync/sync.go`)
Git-based memory sharing using compressed chunks.
#### Architecture
```
.engram/
├── manifest.json ← index of all chunks (small, git-mergeable)
├── chunks/
│ ├── a3f8c1d2.jsonl.gz ← chunk 1 (gzipped JSONL)
│ ├── b7d2e4f1.jsonl.gz ← chunk 2
│ └── c9f1a2b3.jsonl.gz ← chunk 3
└── engram.db ← local working DB (gitignored)
```
#### Workflow
```bash theme={null}
# Export new memories as a chunk
engram sync
# Creates .engram/chunks/{hash}.jsonl.gz + updates manifest.json
# Commit to git
git add .engram/ && git commit -m "sync engram memories"
# On another machine: import new chunks
engram sync --import
# Reads manifest.json, imports chunks not in sync_chunks table
# Check status
engram sync --status
# Shows: 3 chunks in manifest, 2 imported, 1 pending
```
**Why chunks?** Each `engram sync` creates a NEW file — old chunks are never modified. No merge conflicts, just append-only git history.
### 5. TUI (`internal/tui/`)
Interactive terminal UI built with [Bubbletea](https://github.com/charmbracelet/bubbletea).
#### Structure (Gentleman Bubbletea Patterns)
* Screen constants (`Screen int` iota)
* Single `Model` struct holds ALL state
* `Init()` function
* Custom messages for async operations
* Lipgloss styles organized by concern
* Catppuccin Mocha color palette
* Layout, dashboard, list, detail, timeline, search styles
* `Update()` with type switch
* `handleKeyPress()` routes to per-screen handlers
* Each handler returns `(tea.Model, tea.Cmd)`
* `View()` routes to per-screen renderers
* Shared `renderObservationListItem()` for consistent formatting
#### Screens
* **Dashboard**: Stats overview + menu
* **Search**: FTS5 text search with text input
* **Search Results**: Browsable results list
* **Recent Observations**: All observations, newest first
* **Observation Detail**: Full content, scrollable
* **Timeline**: Chronological context (before/after)
* **Sessions**: All sessions list
* **Session Detail**: Observations within a session
TUI uses dedicated store methods (`AllSessions`, `AllObservations`) that don't filter by session status, unlike MCP context methods which only show completed sessions.
## Data Flow
### Observation Creation
```mermaid theme={null}
sequenceDiagram
participant Agent
participant MCP
participant Store
participant SQLite
Agent->>MCP: mem_save(title, content, type, topic_key)
MCP->>MCP: Strip tags
MCP->>Store: AddObservation(params)
Store->>Store: Strip tags (2nd layer)
Store->>Store: Generate normalized hash
alt Topic key provided
Store->>SQLite: SELECT existing with same project+scope+topic_key
alt Existing found
Store->>SQLite: UPDATE observation (increment revision_count)
else No existing
Store->>SQLite: INSERT observation
end
else No topic key
Store->>SQLite: SELECT duplicates in 15min window (by hash)
alt Duplicate found
Store->>SQLite: UPDATE duplicate_count, last_seen_at
else No duplicate
Store->>SQLite: INSERT observation
end
end
Store->>SQLite: FTS5 trigger auto-updates observations_fts
Store-->>MCP: Observation ID
MCP-->>Agent: Success + ID
```
### Search Flow
```mermaid theme={null}
sequenceDiagram
participant Agent
participant MCP
participant Store
participant SQLite
Agent->>MCP: mem_search(query="JWT auth", type="decision")
MCP->>Store: Search(query, options)
Store->>Store: Sanitize FTS5 query (wrap words in quotes)
Store->>SQLite: SELECT FROM observations_fts WHERE ... ORDER BY rank
SQLite-->>Store: Rows with BM25 rank
Store->>Store: Truncate content to MaxObservationLength
Store-->>MCP: SearchResult[] (with rank, truncated content)
MCP-->>Agent: Compact results + metadata
Note over Agent: Agent sees result ID 42 is interesting
Agent->>MCP: mem_timeline(observation_id=42, before=3, after=3)
MCP->>Store: GetTimeline(42, 3, 3)
Store->>SQLite: SELECT observations WHERE session_id=... AND created_at < ...
Store->>SQLite: SELECT observations WHERE session_id=... AND created_at > ...
Store-->>MCP: TimelineResult (focus + before + after)
MCP-->>Agent: Chronological neighborhood
Note over Agent: Agent needs full content
Agent->>MCP: mem_get_observation(id=42)
MCP->>Store: GetObservation(42)
Store->>SQLite: SELECT * WHERE id=42 AND deleted_at IS NULL
Store-->>MCP: Full Observation (no truncation)
MCP-->>Agent: Complete content
```
## Plugin Architecture
### OpenCode Plugin
TypeScript adapter in `~/.config/opencode/plugins/engram.ts`:
Checks if `http://127.0.0.1:7437/health` responds. If not, spawns `engram serve` in background.
Detects `.engram/manifest.json` in project directory, runs `engram sync --import`.
`ensureSession()` creates session on-demand via HTTP API (idempotent, survives plugin reloads).
`chat.system.transform` hook concatenates Memory Protocol into system prompt (compatible with single-system-message models like Qwen, Ministral).
`experimental.session.compacting` hook injects previous context + instructs compressor to call `mem_session_summary`.
Strips `` tags before sending to HTTP API (defense layer 1).
### Claude Code Plugin
Bash scripts + skill system in `~/.claude/plugins/engram/`:
* **startup** (`session-start.sh`): Ensures server, creates session, imports chunks, injects context
* **compact** (`post-compaction.sh`): Injects context + recovery instructions
* **stop** (`session-stop.sh`): Logs session end
`skills/memory/SKILL.md` — Full Memory Protocol loaded into skill context.
`.mcp.json` registers `engram mcp` server automatically.
Claude Code plugin hooks use **bash scripts**. On Windows, requires Git Bash or WSL. Use bare MCP (Option C in setup) for native Windows support.
## Design Decisions
* **Single binary**: No Node.js, no Bun, no runtime dependencies
* **Cross-platform**: Works natively on macOS, Linux, Windows (x86\_64 and ARM64)
* **Pure Go SQLite**: `modernc.org/sqlite` has no CGO — true single binary distribution
* **Fast startup**: Go binary starts in milliseconds vs Node.js \~100ms
* **FTS5 covers 95% of use cases**: Full-text search with BM25 ranking is sufficient for code memory
* **No separate process**: No ChromaDB, no Pinecone, no vector service to manage
* **Single file**: `~/.engram/engram.db` is portable, backupable, debuggable with `sqlite3` CLI
* **Simpler**: Zero configuration, zero network calls
* **Agent already has the LLM**: No need for separate API calls
* **Agent has context**: It knows what just happened better than a background service
* **No extra cost**: No additional API usage beyond the agent's work
* **No latency**: Compression happens inline during the agent's response
* **Raw tool calls are noisy**: `edit: {file: "foo.go"}` pollutes search results
* **Low signal-to-noise ratio**: Shell history and git already provide the audit trail
* **Bloats database**: Thousands of low-value observations vs dozens of curated summaries
* **Agent curates better**: Structured `mem_save` summaries are higher quality and more searchable
## Project Structure
```
engram/
├── cmd/engram/main.go # CLI entrypoint — all commands
├── internal/
│ ├── store/store.go # Core: SQLite + FTS5 + all data operations
│ ├── server/server.go # HTTP REST API server (port 7437)
│ ├── mcp/mcp.go # MCP stdio server (13 tools)
│ ├── setup/setup.go # Agent plugin installer (go:embed)
│ ├── sync/sync.go # Git sync: manifest + chunks (gzipped JSONL)
│ └── tui/ # Bubbletea terminal UI
│ ├── model.go # Screen constants, Model struct, Init()
│ ├── styles.go # Lipgloss styles (Catppuccin Mocha palette)
│ ├── update.go # Update(), handleKeyPress(), per-screen handlers
│ └── view.go # View(), per-screen renderers
├── plugin/
│ ├── opencode/engram.ts # OpenCode adapter plugin
│ └── claude-code/ # Claude Code plugin (hooks + skill)
│ ├── .claude-plugin/plugin.json
│ ├── .mcp.json
│ ├── hooks/hooks.json
│ ├── scripts/ # Bash scripts for lifecycle hooks
│ └── skills/memory/SKILL.md # Memory Protocol skill
├── DOCS.md
├── README.md
├── go.mod
└── go.sum
```
## Dependencies
* `modernc.org/sqlite` — Pure Go SQLite (no CGO)
* `github.com/mark3labs/mcp-go` — MCP protocol implementation
* `github.com/charmbracelet/bubbletea` — Terminal UI framework
* `github.com/charmbracelet/lipgloss` — Terminal styling
* `github.com/charmbracelet/bubbles` — TUI components
* `@opencode-ai/plugin` — OpenCode plugin types
* Runtime: Bun (built into OpenCode)
* Claude Code plugin: bash (for hooks)
## Next Steps
Memory system, session lifecycle, and 3-layer pattern
When to save, when to search, and session close protocol