When people say "stop just prompting and build an agent harness," they're usually talking about separating the AI model from the software around it.
The model is good at reasoning in the moment. The harness is what gives it memory, tools, workflows, and rules.
For a solo builder, this doesn't need to be complicated. A good harness is often only a few hundred lines of code.
Here's the mental model:
User
│
▼
Agent Harness (your code)
┌──────────────────────────┐
│ Load memories │
│ Load project context │
│ Choose tools │
│ Search docs │
│ Call LLM │
│ Save new knowledge │
└──────────────────────────┘
│
▼
LLM API
The important point is that the LLM is stateless. Every API call starts fresh. The harness decides what information to provide each time.
The four things every useful harness has
1. Project memory
Instead of relying on one giant chat, store information about each project.
Example:
projects/
startup-a/
summary.md
architecture.md
decisions.md
todos.md
meetings/
docs/
client-b/
summary.md
api.md
roadmap.md
Before asking the model a question, your harness loads:
- project summary
- current task
- recent work
- important decisions
instead of sending the entire conversation history.
2. Long-term memory
Not everything belongs in the context window.
For example:
User prefers FastAPI.
User hates Redux.
Current server is on Hetzner.
Stripe account uses API version 2025-04.
Always use Ruff formatting.
Store these separately.
When a new request arrives:
"What should we build next?"
your harness retrieves relevant memories.
This is Retrieval-Augmented Generation (RAG), but applied to your own work instead of general documents.
3. Session memory
Keep only the current conversation.
Example:
Today:
Investigated login bug.
Changed OAuth callback.
Need to test Safari.
Need migration tomorrow.
After a few hours, summarize it.
Instead of storing:
15,000 messages
store:
Summary:
• OAuth callback fixed
• Cookie issue remains
• Safari testing pending
Now future chats stay compact.
4. Tool layer
The model shouldn't pretend to know things it can look up.
Give it tools like:
Read file
Write file
Search code
Git diff
Run tests
Run SQL
Open browser
Search documentation
Create issue
Update TODO
The harness decides which tools are available and executes them when requested.
What "remember previous projects" actually means
A common beginner mistake is thinking the model "remembers."
Instead:
User:
Continue the CRM project.
The harness does something like:
Load:
CRM summary
Recent commits
Architecture notes
Decision log
Open tasks
Relevant documents
Then constructs a prompt:
System:
You are helping with the CRM project.
Project Summary:
...
Architecture:
...
Recent changes:
...
Open tasks:
...
User:
Continue implementing permissions.
The model now appears to "remember," but it's because your software supplied the context.
How I'd organize a solo-builder harness
memory/
projects/
crm/
summary.md
decisions.md
roadmap.md
changelog.md
ai-assistant/
summary.md
architecture.md
people/
alice.md
bob.md
personal/
preferences.md
coding-style.md
scratch/
logs/
Every interaction updates these files.
The workflow
Imagine you ask:
Continue working on my SaaS dashboard.
The harness automatically:
1. Detect project
↓
2. Load project summary
↓
3. Load recent commits
↓
4. Search notes
↓
5. Search previous conversations
↓
6. Retrieve related memories
↓
7. Build prompt
↓
8. Call model
↓
9. Save new decisions
↓
10. Update summaries
You never manually paste context.
Memory should be layered
Think in different time scales.
Permanent
- coding preferences
- API keys (or references to secure storage, not the secrets themselves)
- favorite libraries
↓
Project
- architecture
- roadmap
- decisions
- documentation
↓
Weekly
- progress
- blockers
- meeting notes
↓
Current Session
- current task
- today's conversation
Only the relevant layers get loaded.
Use summaries instead of raw chats
Instead of storing:
300 pages of conversation
store:
Project Summary
Authentication completed.
Billing uses Stripe.
Database migrated to Postgres.
Known issues:
- OAuth timeout
- Missing analytics
Next priorities:
1. Teams
2. Audit logs
3. Metrics
That's often enough for the model to pick up where it left off.
A practical tech stack
You don't need an elaborate framework to get started. A lightweight setup could be:
- A small Python or TypeScript application as the orchestrator.
- Markdown or JSON files (or SQLite) for persistent memories.
- A vector database only if semantic search across many notes becomes valuable.
- Git for tracking code and project history.
- An LLM API for reasoning and generation.
Many developers start with plain files and SQLite and only add vector search when they have hundreds or thousands of notes.
Keep memory curated
Don't save every exchange forever. Instead:
- Save durable decisions ("We chose PostgreSQL because…").
- Save stable preferences ("Prefer FastAPI and Ruff.").
- Summarize completed sessions.
- Discard temporary back-and-forth once it's been distilled into useful knowledge.
This keeps retrieval focused and avoids repeatedly surfacing outdated or contradictory context.
The biggest shift is thinking of the model as a reasoning engine rather than a database. The harness becomes the operating system around it: it decides what context to load, what tools to expose, what memories to retrieve, and what new knowledge to save. With that approach, you can stop depending on one ever-growing chat and instead have agents that can reliably pick up work across multiple projects and over long periods.
