Answer multi-step questions without stitching chunks
"Who approves a refund over $5,000?" is three facts, not one. There is a refund policy, a dollar limit, and an approver role, and they usually live in three different places in your documents. A human reads across them without thinking. A vector search does not. It returns the passages most similar to the question and leaves the joining to your agent, which is where multi-step answers quietly fall apart.
This is the second thing that breaks after a retrieval-augmented agent ships. The simple questions already worked in the demo. The real ones are almost all multi-step, and they are the ones that go wrong.
Why similarity search struggles with two hops
Embedding search is good at "find me passages about refunds." It is weak at "connect the refund rule to the limit that governs it to the person who signs off." Nothing in a flat vector index records that those three things belong together. So teams reach for workarounds:
- Multi-query retrieval: ask several sub-questions and merge results. This can miss the join and it multiplies latency and cost.
- Agentic re-search loops: let the agent search, read, and search again. Powerful, but slow, non-deterministic, and hard to debug when it loops.
- GraphRAG: build a knowledge graph first. It genuinely helps multi-hop, but the indexing pass sends your corpus through an LLM, which is why the complaints about rebuild cost are so common.
All three are trying to recover a structure that the source documents already implied but the storage layer threw away.
Keep the connections instead of rebuilding them
FastMemory stores your domain as a map with six plain layers rather than a flat cloud:
- What it's made of
- How it's organized
- What it does
- What data it uses
- Who is allowed
- What happens when
The refund question walks straight across them: the rule (what it does) points to the limit (what data it uses) which is gated by a role (who is allowed). Those links are written down when the map is built, so answering a three-step question is a traversal, not a lucky re-rank. You get the connected answer and the route it took, on the first try, without prompt engineering to coax the agent into searching again.
You write the connections in ATF Markdown, a plain format with one section per thing:
## [ID: refund_policy]
**Action:** Approve or decline a refund request
**Data_Connections:** refund_limits, orders_table
**Access:** support_agent, finance_lead
**Events:** refund_approved, refund_declined
## [ID: refund_limits]
**Action:** Define approval thresholds
**Data_Connections:** finance_policy
**Access:** finance_lead
Data_Connections and Access are the joins. FastMemory reads them at build time with no LLM call, so keeping the structure costs you nothing per document and nothing per rebuild.
Run the multi-hop question yourself
cargo install fastmemory
fastmemory build data/input.md
fastmemory query data/input.md "who approves a refund over $5,000?"
The answer comes back with the trail across the layers, so you can confirm it connected policy, limit and role rather than stitching together three unrelated passages. Point it at a two- or three-hop question from your own domain that a flat retriever gets wrong today, and compare.
If you want the map serving an IDE agent instead of the CLI, fastmemory mcp data/input.md exposes it over MCP for Claude Code or any MCP-compatible client.
An honest note on numbers
You will find plenty of latency and recall figures floating around for memory systems, including some we have seen misquoted about ours. The claim worth making here is structural, not a benchmark: FastMemory keeps the connections your documents already implied, so multi-hop answers are a traversal you can inspect. FastMemory is open source and MIT licensed, and it reports state-of-the-art results on 13 public benchmarks, so you can reproduce the retrieval-quality claims rather than take them on faith. Test it on your own hard questions before you believe any single number.
Where to start
- See the six layers and how the joins are stored on the FastMemory architecture page.
- Clone the worked example from GitHub.
- Write one ATF Markdown file that connects two facts you care about, and ask the question that needs both.
Multi-step answers do not need a bigger model or a longer prompt. They need a memory that remembers how your facts connect.