Building an AI assistant on your company knowledge

Ask a new starter how they find the current expenses policy and you will hear the real answer: they ask the colleague who sits nearest. The document exists, probably in three versions across two systems, and nobody is confident which one is live. That is the problem a knowledge assistant is meant to solve, and it is a content problem as much as a technology one.

What is an AI knowledge assistant?

An AI knowledge assistant is software that answers employees' questions from your company's approved documents and shows the source of each answer. It is not a general chatbot: it looks up your policies, procedures and records first, then writes an answer from what it finds.

What is retrieval-augmented generation, in plain English?

Retrieval-augmented generation, usually shortened to RAG, is the pattern behind almost every company knowledge assistant. AWS describes it as "the process of optimizing the output of a large language model, so it references an authoritative knowledge base outside of its training data sources before generating a response" (AWS).

In practice it works in four steps.

1. Index. Your approved documents are split into passages and stored in a way that supports search by meaning as well as by keyword.
2. Retrieve. When someone asks a question, the system finds the passages most likely to contain the answer, filtered by what that person is allowed to see.
3. Ground. Those passages are supplied to the model alongside the question, with instructions to answer from them.
4. Cite. The answer comes back with links to the source documents, which AWS notes lets users verify information themselves and increases trust in the result.

Two consequences follow, and both matter more than the choice of model. Accuracy is a function of the sources, so a well written policy indexed correctly beats a larger model reading a stale one. And nothing is being trained, so when a document is corrected the answer changes as soon as it is reindexed.

Microsoft 365 Copilot vs Gemini for Workspace vs Glean vs a custom knowledge assistant: which fits a 20 to 200 person company?

For a company of 20 to 200 people, buying is the right answer more often than suppliers admit, and a custom assistant earns its cost only when the knowledge sits outside one mainstream suite. Use the table to see which situation you are in, then check the vendor notes and list prices below it.

• Microsoft 365 Copilot (Microsoft now calls it Microsoft Copilot) suits organizations whose knowledge already sits in Microsoft 365, because it works within your tenant and inherits Microsoft Graph permissions.
• Gemini for Workspace is the equivalent for Google Workspace customers, now included in the plans rather than sold separately.
• Glean and similar enterprise search platforms connect many systems at once. Glean describes itself as "an AI platform for work that unifies enterprise search, assistants, and agents on your company's context" (Glean).

The mixed answer is common and perfectly reasonable: licensed assistants for general documents, and a custom assistant for the one domain where the answers live somewhere the suite cannot reach. For the broader picture of where retrieval fits alongside other automation, read the practical guide to AI workflow automation.

Which sources should the assistant be allowed to read?

The instinct is to connect everything. Resist it. Indexing every folder in the company is how an assistant begins quoting a 2021 policy that was replaced twice.

Work through three tiers.

• Authoritative. Current policies, approved procedures, the live price list, signed contracts, the product catalog. These are the sources the assistant should answer from, and each one needs a named owner and a review date.
• Useful with context. Past project documentation, meeting notes, resolved support tickets. Valuable for background, but they should not be quoted as policy.
• Excluded. Drafts, personal drives, anything superseded, and anything whose sensitivity means it should never appear in an answer.

Then do the unglamorous part. Retire old versions rather than leaving them beside the new one. Give each source a date. Fix the document titled "Final v3 (updated)" that three teams reference differently. Most disappointing assistant pilots trace back to this step being skipped, not to the retrieval.

How do you stop an AI assistant showing people documents they should not see?

An AI knowledge assistant stays safe by enforcing one rule: a person must never see through the assistant anything they could not open directly. Enforcing it means filtering at retrieval, using the asking user's own identity, rather than filtering the answer after the fact.

Microsoft describes exactly this model for Microsoft 365 Copilot, now called Microsoft Copilot. It uses Microsoft Graph to reach data "in the user's unique context", and states that "Copilot only accesses data that an individual user is authorized to access" and "can't access data that the user doesn't have permission to access", while honoring Conditional Access policies and multifactor authentication (Microsoft Learn). A custom assistant has to reproduce that behavior across whatever systems it connects to, which is usually the hardest engineering problem in the project.

Three checks belong in the plan.

1. Test with real roles. Sign in as accounts across different departments and seniority levels and confirm that results differ correctly.
2. Audit for existing oversharing. Search makes findable what was previously merely present. A confidential file in an open folder was always a problem, but nobody could find it before.
3. Log every query and source returned. So a question about what the assistant disclosed has an answer.

What do citations prove, and what do they not?

Citations are the single most useful feature of a knowledge assistant, and the most misread. A citation shows which passage the answer was drawn from, so a reader can check it. It does not prove the answer is a correct reading of that passage, and it does not prove the passage is current.

Design for checking rather than trust. Link to the specific section or page rather than the document. Show the retrieved text alongside the answer, so the reader can compare in one glance. Display the document's date and owner, because an answer sourced from a superseded policy looks identical to a correct one. Where an answer draws on several documents, attribute each claim separately.

What should happen when information is missing or sources conflict?

Most knowledge assistants are designed for the case where the answer exists in one place, so the missing, conflicting and out of scope cases need deliberate design.

Missing. When retrieval returns nothing relevant, the assistant should say so plainly and point to who owns that subject, rather than filling the gap from general knowledge. Test this deliberately by asking questions your documents do not cover. An assistant that answers everything confidently is worse than one that declines.

Conflicting. When two documents disagree, the assistant should show both and say which is more recent or more authoritative rather than silently choosing. In practice a conflict is a content defect, so route it to the source owner as a task. Over a few months this turns the assistant into a way of finding stale documentation, which is worth as much as the answers.

Out of scope. Decide where the boundary sits. Legal interpretation, HR casework and anything requiring professional judgment should return the relevant material and the right contact, not an opinion.

The OWASP Top 10 for LLM Applications 2025 names misinformation, sensitive information disclosure and prompt injection among the leading risks for this kind of system (OWASP). A knowledge assistant touches all three, which is why source control, permissions and refusal behavior are the design, not the polish.

Does a customer-facing assistant change the requirements?

A customer-facing assistant needs stricter controls than an internal one, because the mechanism is the same while the tolerance for error is not: a customer acts on a wrong answer or screenshots it.

A common and sensible sequence is to launch internally first, use several months of real questions to find the gaps in your content, then expose a narrower, tested subset to customers.

How do you evaluate an assistant before and after launch?

Evaluating a knowledge assistant starts with a test set, because impressions from a demo tell you almost nothing.

1. Collect real questions. Fifty to two hundred, gathered from the service desk, the shared inbox and the people who get interrupted most.
2. Write the correct answer and the source for each. This is the work. It is also a content audit you would benefit from anyway.
3. Include the awkward ones. Questions your documents do not answer, questions where two documents disagree, and questions that should be refused.
4. Score each answer on four things. Is it correct, is the cited source right, is it complete, and did it decline when it should have?
5. Rerun the set on every change. New sources, a new model version, a changed prompt. Quality moves in both directions.

After launch, track usage rather than assuming it. Measure how many people ask a question in a given week, the share of answers marked useful, how often people click through to the source, and which questions return nothing. That last measure is the roadmap for your content.

Illustrative example: Consider a professional services firm whose consultants repeatedly ask the same questions about expense limits, engagement letter templates and which insurer covers which client type. The firm indexes three sources only: the current staff handbook, the approved template library and the client onboarding procedure, each with a named owner and a review date. Permissions are inherited from SharePoint, so a consultant cannot retrieve a document they could not open. A test set of 120 real questions is built from the operations inbox, including fifteen the documents do not cover. The assistant is launched to one practice group, and the questions that return nothing are reviewed monthly and turned into documentation tasks.

For testing method in more depth, see how to test AI before it touches your operations, and for the access and retention decisions, data security and governance for AI in the enterprise.

How Kastling approaches company knowledge assistants

Company knowledge and search is part of our AI Integration & Automation service: helping employees find and synthesize information from approved internal sources, with source references and suitable access rules. A discovery call is free and covers which questions people actually ask, where the answers live today and who owns them.

Before anything is built we agree which sources are authoritative, how permissions will be enforced and what should happen when information is missing, conflicting or uncertain. We test against real questions with known answers, involve the people who will use it, and agree data access, retention and ownership up front. Source references support review; they do not guarantee that every generated answer is correct, and we would rather say that plainly than promise otherwise.