Knowledge bases rot because storage gets treated as the whole job. Emerjent treats retrieval, ranking, context assembly, and lifecycle as the job: hybrid search that reads meaning and wording together, multi-signal ranking with calibrated relevance tiers, an exact-lookup path that puts canon in front of every agent run, atomic facts that supersede instead of duplicate, per-user isolation inside a shared workspace, and decay into a reversible cold archive with every structural change reviewed before it lands.
If you run agents against an agency's institutional record, the question that decides whether they help or hurt is not "is it stored." It is "when twenty things match, does the right one rank, does the consumer know how much to trust it, and does it reflect what is true today." Get that wrong and every agent downstream, drafting proposals, prepping meetings, answering client questions, inherits the mistake: more revision cycles, more rework, less trust in what the agent hands back. The rest of this article is how Emerjent answers that, mechanism by mechanism.
Why knowledge bases rot
The failure is rarely dramatic, and nothing gets deleted. The knowledge base just keeps accepting writes: session notes, meeting summaries, drafts, research. Each one mentions the same client names, the same product terms, the same keywords as the canonical documents. Retrieval that ranks on keyword density or raw embedding similarity has no way to tell the difference between the document that holds an engagement's actual terms and the twelve notes that mention those terms in passing.
So the rot accumulates quietly. A client overview written in January keeps surfacing in June after the terms changed. Session notes bury the playbook they were derived from. Dead client intel outranks current intel because nothing ever demoted it. And nobody audits what retrieval actually returns, because the answers look plausible right up until a person, or an agent acting for them, does something with a stale one.
Agencies feel this faster than most businesses. Rates change mid-year, contacts turn over, scope shifts between phases, and the volume of notes grows with every engagement. When agents run against that knowledge base to draft proposals, prep meetings, or answer client questions, retrieval quality stops being a convenience and becomes operational risk.
Storage is the easy part
Most products called knowledge bases are storage with embeddings: a folder tree, an upload button, a vector index. That covers writing things down. It says nothing about what happens to those writes over the following year.
A memory system is a different scope of job. It has to answer:
- Which result should rank when twenty match?
- How much should the consumer trust each one?
- What must be in front of an agent every single run, regardless of what today's query embeds near?
- What happens when a stored truth changes?
- What fades, what is protected from fading, and what gets archived?
- Who reviews the changes the system makes on its own?
Emerjent treats those questions as the product. The sections below walk the mechanism: retrieval, ranking, context assembly, atomic facts, scope, consolidation, and lifecycle.
Retrieval that reads meaning and wording together
Two legs, one ranked list
Every search runs two retrieval legs in parallel: a semantic leg over vector embeddings and a lexical leg over full-text indexes. The results fuse into a single ranked list.
Run both because each fails where the other wins. Semantic search handles paraphrase well ("what do we charge them" finds the rate card) but can miss rare exact terminology: an internal codename, a configuration key, a person's name. Lexical search nails exact terms and has no concept of meaning. Microsoft's benchmarks on Azure AI Search found the same pattern at scale: fusing keyword and vector retrieval surfaced relevant results more reliably than either method alone.
The fusion is weighted, and the weights are deliberate. Each leg contributes to a fused score, the lexical leg carrying the larger share and the vector leg the smaller, with a fused-score floor below which a candidate drops out entirely. The vector leg also has its own similarity threshold, so weak embedding matches never enter the pool in the first place. Two details carry most of the protection past that. A result that appears only on the lexical leg enters with a capped contribution, so a keyword-stuffed note can't outrank a real semantic match. And a result that clears neither the vector threshold nor the fused floor is simply gone. Every one of these knobs, the leg weights, the fused floor, the vector threshold, and the tier thresholds below, is per-organization configuration, live-tunable at runtime rather than baked into a deploy and read through a short-lived in-process cache.
The net effect: a well-worded query doesn't come back empty just because one leg found nothing, and repeating the right words isn't enough to win.
Calibrated relevance, not bare scores
Every result carries a relevance tier: high, standard, or low, computed on the fused score before strength ranking touches it. Raw similarity scores compress into narrow ranges that mean different things from query to query; a tier is a calibrated statement about how seriously to take the match. Consuming agents use it to decide whether to act on a result, qualify it, or keep looking. Humans reviewing a retrieval get the same signal.
Graceful degradation
When a retrieval dependency is unavailable (an embedding provider outage, for instance), search falls through modes in order: hybrid, then vector only, then lexical only, then plain substring matching. The response records which mode answered, and the access log keeps that record, so degraded retrievals are queryable after the fact instead of silently shaping answers while nobody is looking. A search that quietly dropped to substring matching for an hour is a thing you can find later, not a mystery.
Strength: deciding what deserves to surface
Relevance measures how well a result matches the query. Strength measures how much the system should trust the memory itself. Final ranking multiplies the two: the fused relevance score times a live strength scalar. A weak-but-canonical document and a strong-match-but-stale note get sorted by the product, not by relevance alone.
Strength is multi-signal, and the signals are stored as a vector, not a single number: recency, how often a resource is retrieved, how many distinct kinds of queries reach it, its conceptual centrality, a confidence value, when it was last validated, and the trust of its source. One signal can't run the show. A document isn't canon just because it's new, and it isn't expendable just because it's old. The vector is kept intact in storage even though ranking collapses it to a scalar, because once you collapse the components in storage you can never recover them, and every future change to the ranking policy needs them.
There's a research basis for ranking this way. Anderson and Schooler's 1991 study showed that the probability a human memory will be needed tracks the recency and frequency of its past use in lawful, predictable ways. A ranking layer that ignores those signals isn't neutral; it just substitutes a worse policy, usually newest-wins.
Decay that spares deep knowledge
Memory strength decays with disuse. Ebbinghaus measured the human version in 1885, and a 2015 replication confirmed the same curve holds: retention drops steeply at first, then flattens into a long tail.
Decay in Emerjent follows a hybrid curve with two summed components: a fast exponential term that erodes quickly, plus a slow power-law tail that barely moves over long horizons. A single fast curve erodes exactly the wrong things. A meeting summary from last week should fade. A playbook that still governs how three client engagements run should not. The long tail is where institutional knowledge lives, so the long tail decays slowly, and the strength scalar that ranking uses is computed live from the stored vector with this decay applied since the memory was last validated.
Floors for the canon
Some memories shouldn't be left to win on signals alone. Architecture decisions, frameworks, and playbooks carry class-based strength floors: at retrieval time, their effective strength can't fall below a configured minimum, and that protection applies automatically from the resource type, with no extra step. Canonical documents that don't fit a protected class (a master CV, a voice guide) can be pinned evergreen, which applies the same kind of floor explicitly. Either way, fresh chatter can't bury the canon, no matter how much of it accumulates.
No rich-get-richer
Because frequency is a strength signal, there's a feedback loop waiting to happen: the most-retrieved documents strengthen, rank higher, and get retrieved even more, until a handful of documents monopolize every answer. Retrieval applies diversification (maximal marginal relevance) after strength ranking, trading a little redundant similarity for coverage of the candidate pool. The strongest documents still surface; they just can't crowd out everything adjacent to them.
Two paths into every agent run
Search is what an agent calls when it goes looking. But some things an agent needs shouldn't depend on it looking, and shouldn't depend on whether tonight's query happens to embed near them. A governance policy that must always hold, the org's resolved vocabulary, the one canonical document a task cannot run without: retrieving those by similarity is a bug. A policy is not more or less in force depending on cosine distance.
So agent-run context is assembled along two structurally separate paths before every run, and they are kept apart on purpose.
Path A is exact enumeration, and it runs unconditionally. No embeddings, no ranking, just direct lookups: the org's active governance policies, rendered compactly; the resolved variable hierarchy (organization, then client, then deal) that carries the methodology vocabulary; and pinned canon, read whole from the durable content of each resource rather than from its chunks. Chunks are a retrieval index. Pins are canon, and canon gets injected in full. Path A is added first and it is not droppable: if the budget is tight, canon does not get cut.
Path B is semantic discovery, and it's opt-in. When an agent is configured for it, ranked search results keyed on the run's instruction are pre-fetched and appended, whole-item in rank order, filling whatever budget remains after canon. An agent can always still call search as a tool mid-run regardless of this flag; Path B is just the pre-fetch.
The hard rule is structural, not a convention: semantic results never enter the static system-prompt prefix. The assembler hands back two separate strings. One is the stable prefix (Path A). The other is a volatile tail appended to the user prompt under its own header (Path B). Only the prefix reaches the system prompt, and the separation is enforced by the type contract and the call sites. That ordering is also what makes provider-side prompt caching pay off: the same agent, org, pin set, and variable set produce a byte-stable prefix across runs, so the slowly-changing canon sits in cache while only the volatile tail changes.
There's a budget model underneath it. Canon is added in priority order and never truncated mid-record, because a half-injected memory is worse than an absent one; an oversized single pin is cut at a paragraph boundary and flagged. Semantic items fill the remainder whole, and the lowest-ranked whole item drops first when space runs out. Every run stamps a trace of exactly what canon was in force: the budget, which pins injected, which were truncated, the variable and policy counts, whether Path B was on, and what it pulled. You can answer "what did this agent actually see" without re-deriving it.
The practical payoff is small to describe and large in effect. Whether the agent drafting a proposal sees the master CV, or the voice rules, or the governance policy that gates the send, no longer depends on embedding luck. It is enumerated, every run, by construction.
Facts, not just documents
Some truths are smaller than a document: a renewal date, a billing rate, a preferred contact channel. Stored inside prose, these are exactly the values that go stale invisibly. The document gets updated, or doesn't, and older copies in other documents keep matching queries.
Emerjent stores these as atomic facts: a subject, a relation, and one self-contained sentence, with provenance pointing back to the document or run the fact was distilled from. The structure makes change handling exact. When a new fact lands on the same subject and relation with different content, the old fact is revoked and pointered to its replacement in the same transaction that writes the new one. There is no window where both versions rank. Identical content dedups against the existing fact (matched on a normalized content hash) instead of writing a copy.
The write path is gated, and the gate is a role-and-scope matrix, not a free-for-all. A fact added by an owner or admin lands active. A fact added by a solutions engineer on a deal it's scoped to lands active; a project manager adding a client-scoped fact lands active. Anything outside that matrix, and every fact the nightly consolidation cycle extracts on its own, lands provisional. Provisional facts stay out of recall entirely until someone promotes them through the review queue. Status is computed server-side; a caller cannot push a fact straight into recall by claiming it's already trusted.
Fact search runs through the same hybrid retrieval and relevance tiers as documents. The unit of truth that changes gets superseded, not duplicated.
Atomic facts sit alongside document-grained resources, not instead of them. Documents stay the home for anything with narrative or structure; facts carry the single-sentence truths that change on their own clock.
Private by user, not just by tenant
A knowledge base shared across an agency has a quieter risk than rot: one person's context leaking into another's. Every workspace is already isolated by organization at the database level. Inside an organization, memory adds a second axis, and it stops being theoretical the moment a second person joins an org.
Facts and the pins that inject a person's standing context carry a user scope. Left unset, a fact is shared across the org and behaves exactly as before. Set to a user, it becomes a private partition that no teammate sees, including org admins and owners. That partition runs deeper than a visibility filter, because the user scope joins the keys that decide dedup and supersession, not just the ones that decide reads. Two people can hold the same assertion about the same subject with different content, and each supersedes only their own; a private fact can never collide with, dedup against, or overwrite a shared one, or another user's private one. Keeping the scope out of the write keys would have let a user-private fact quietly supersede an org-shared fact on the same subject and relation, which is interference at the write layer, not just at retrieval. That is why the partition is a key, not a filter.
The privacy guarantee is enforced in two layers because one is not enough. The primary guard is a visibility predicate in the worker's own SQL, because the worker runs with service-role access that bypasses row-level security, so it has to carry the predicate itself. The backstop is the row-level-security user predicate, verified with impersonation tests. Both are required. User-private memory is fully private: invisible to everyone but its owner in retrieval and in list and search, admins and owners included. Admins still fully manage shared memory; they simply cannot read another person's private partition.
When the system resolves what to put in front of an agent, it works outward from the person for the user-scopable classes: user, then deal, then client, then org, with shared memory always co-retrieved alongside the user's own. A user-scoped pin orders ahead of a shared pin in the same slot; a user-scoped fact ranks above a shared fact of equal relevance. Tenant-shared truths like client intel and deal facts stay org, client, or deal scoped; they are not promoted into a person's private space. The effect is that you can run every client and every teammate in a locked room, on the same knowledge base.
A working session (a single chat or run) keeps its own scratch context, keyed to that run and ephemeral by design. It does not bleed into the next session unless you deliberately promote it into durable memory, and promotion is an ordinary write, so nothing carries forward by accident. This is the same relationship an episode has to the session it came from: the episode is the promoted, durable summary; the working memory is the pre-promotion scratch. They don't duplicate. Promotion is the bridge.
The consolidation cycle: propose, never write
Left alone, a growing knowledge base needs housekeeping: a distilled claim that has earned a place in durable memory, a saturated low-value document that should step down in rank. Emerjent runs that housekeeping on a schedule (nightly, in the small hours), but it never writes structural changes silently. It proposes them.
Passive consolidation is the first and shipped loop of a larger design. It scores candidates and files each proposed change to a review queue where a person records a decision: keep, or revert. Revert restores the exact prior state, down to a demoted document's pre-cycle strength snapshot. Promotion and demotion are symmetric; both are reviewable, both are reversible.
What gets proposed follows a gate you can read and tune, not a model's mood on a given night. The gate is a pure function over a candidate's signal vector, and it runs one of two ways per workspace: a blended-score threshold, or a k-of-n rule that fires when any k of the signals clear their bars. The split between what the model decides and what the record decides is the important part. The signals that can be counted, how often something actually came up and how many distinct questions it answered, are computed straight from the run's cited supporting activity. The model is left only the signals that need judgment: how central a fact is, how confident it is, whether it's worth keeping at all. An offline sweep over a labeled set is what set the default threshold, and it landed roughly equivalent to a moderate k-of-n rule and well ahead of the loosest "any single signal" gate, which over-promotes. Crucially, the same blended score that clears the gate becomes the memory's starting strength, so the number that decided promotion is the number that ranks it later. A rare-but-important fact that surfaced once can still earn its place on the strength of its other signals; routine chatter doesn't get promoted just because it repeats.
Consolidation isn't only passive. A second shipped loop does active, directed work on memory: spaced rehearsal on a Leitner-box schedule, with high-stakes memories auto-enrolled, so the things that matter get re-validated on a cadence instead of decaying untouched. A third loop is the access-based strengthening already described above, the live strength dynamics that run continuously rather than on a schedule. Two further loops (creative recombination, and distilling consolidated knowledge back into model weights) are on the roadmap, not shipped, and are called out here as roadmap on purpose.
The review surface has one more layer worth naming: trust calibration. As a workspace builds a track record of keeping a given kind of change (say, framework promotions kept nineteen times out of nineteen), low-risk changes of that shape can be auto-kept, with the human-only keep-versus-revert stats kept separate so the trust signal never becomes circular, and any auto-keep individually undoable back into the human queue. The system earns the right to stop asking about the boring cases, and you can take that right back one decision at a time.
Forgetting on purpose
Decay has a destination, and it isn't deletion. When a memory's strength falls far enough, it moves to a cold archive: out of everyday recall, still retrievable on request for audits or restoration, and fully reversible. Everyday search leaves the archive out; an audit flag pulls it back in, and restoring an archived memory puts it back in active recall as if it had never left.
Hard deletion happens only on explicit user action, never as a side effect of decay. Even then, the audit skeleton survives: the memory's content is redacted from its history, but the record that it existed, when it changed, and how its strength moved over time remains, because that audit trail is deliberately decoupled from the resource it describes and outlives it. You can erase what a memory said without losing the fact that you once knew it. A stricter erasure mode drops the resource name from the final record too, for the cases where even the name is sensitive.
Built and run on a real practice
Emerjent's memory system runs the consulting practice it was built inside, which means the failure mode this article describes happened here, with this system watching.
A canonical client overview, the document holding an engagement's actual terms, had been typed as a generic document with no evergreen pin. On a query about those terms, it was the top lexical match. It still lost the final ranking, because session notes and playbooks that merely mentioned the same keywords carried strength floors the overview didn't. The source of truth was buried by its own metadata.
The fix was one correction, applied through the same tools any user has: the right resource type plus an evergreen pin. The overview came back to the top with the current terms. Two things make that worth telling. The ranking layer made the failure legible, because the result carried its scores and the diagnosis was readable instead of mysterious. And the repair was a single reviewable metadata change, not a re-architecture.
The legibility isn't a one-off. Every retrieval in the system stamps a trace of how it answered: which legs fired, how many candidates fused and under what weights, where the relevance floor cut, and that the diversification pass ran. When a wrong thing surfaces, you get to see why it ranked, which is the difference between fixing retrieval and guessing at it.
Where this lives in the product
The memory system described here sits under Emerjent's knowledge base and under every agent that retrieves from it or gets its context assembled before a run. The architecture overview is at /technology. The other half of this system, how the platform learns from corrections and consolidates what it learns, is the Learning deep dive. The same review-first posture applied to agent actions is covered in Governance, and the knowledge base's role as an agency's institutional record runs through Agency Operations.