Private knowledge assistant vs shared AI tool: which does your firm need?

What is the difference between a shared AI tool and a private knowledge assistant?

A shared AI tool is a vendor’s business plan, such as ChatGPT Business or Enterprise, Claude Team or Enterprise, Microsoft Copilot or Gemini, used on a service the vendor runs for many customers under its own terms. A private knowledge assistant is set up for one firm. It answers from an approved set of the firm’s documents, runs where the firm or its provider chooses, and the firm decides who can ask what and what is logged.

The line between them has blurred, and a fair comparison starts there. As at September 2026, Microsoft Copilot (formerly Microsoft 365 Copilot) answers licensed users from emails, chats and documents in Microsoft 365 that each user already has permission to see. ChatGPT Business connects to internal sources through apps that respect existing permissions, and Claude’s Team plan includes enterprise search across your organisation. So the real question is not whether AI can read your documents. It is who controls where they go, what the assistant may answer from, and who looks after it.

When is a shared business AI tool enough?

A shared tool is enough when most of the value comes from general drafting and summarising, and your rules allow the data involved. For many 10 to 100 person firms it is the right first step, and it is quicker and cheaper to start.

It usually fits when:

• Staff mainly draft emails and letters, summarise public material, reword documents and prepare meeting notes.
• The work can be done without client identifiers, or your policy and client terms allow client information in the tool.
• Your documents already sit in Microsoft 365 with permissions you trust, so Copilot’s answers stay within what each person may see.
• Someone will configure it properly: single sign-on, training and retention settings, approved connectors and a usage policy.

The vendors’ own documentation helps. As at September 2026, OpenAI, Anthropic and Microsoft each state that business customers’ content is not used for model training by default. Processing location is less simple. Microsoft says customers outside the EU may have Copilot queries processed in the US, the EU or other regions. OpenAI says that when a ChatGPT Business workspace stores content in a region outside the United States, a copy of every prompt and response is still kept in the United States for a limited time for abuse monitoring (OpenAI help centre).

None of that rules a shared tool out. It means the firm should decide, in writing, which data may go into it. Our guide to setting up ChatGPT Business securely covers the settings that matter.

When is a private knowledge assistant worth it?

A private assistant is worth it when the firm needs control a shared tool cannot give: over which documents answers come from, where client data is processed, and who can see what. One strong reason from the list below is usually enough to justify scoping one.

1. Client data cannot go into a shared service. Client terms, the sensitivity of the work (family, health, criminal or employment matters) or firm policy rule it out.
2. Answers must come from approved sources only, such as the precedent bank, procedures manual and practice notes, with a citation to the document.
3. You need to choose where data is processed and which model is used, and keep that choice when vendors change their products.
4. Different teams need different content, with an audit log of who asked what.
5. The same internal questions keep landing on senior staff.

For example, picture a 40-person commercial law firm that keeps precedents, file-opening procedures and practice notes across SharePoint and Actionstep. Junior lawyers regularly ask senior associates which lease precedent to use or what the file-opening checklist requires. A private assistant over the approved precedent bank and procedures manual answers with a link to the source document, and says so when the answer is not in the sources. Senior associates stop acting as the firm’s search engine.

For law firms, our comparison of private AI and ChatGPT goes further into confidentiality and client terms.

How does a private knowledge assistant answer from your documents?

Most private assistants use retrieval-augmented generation (RAG). The system looks up relevant passages in your approved documents at the moment a question is asked, and the model answers from them. The model is not retrained on your files.

In practice it works in five steps:

1. Choose and clean the sources. Remove superseded precedents and duplicate policies first. The assistant is only as current as its documents.
2. Index them. Documents are split into passages and indexed so they can be searched by meaning, not just keywords.
3. Retrieve. When someone asks a question, the system finds the most relevant passages and, in a well-designed system, only from documents that person may see.
4. Answer with citations. The model writes an answer from those passages and links to each source, so a lawyer or accountant can check it.
5. Log and review. Questions and answers are logged, so the firm can see what people ask and where answers fall short.

The privacy difference is real. The OAIC’s guidance on developing and training generative AI models says that using personal information you already hold to train or fine-tune a model needs careful analysis under APP 6, often with consent or an opt-out. Retrieval avoids training, and a document removed from the index stops being used. The Privacy Act still applies to any personal information the assistant handles, so the design needs the same care as any other client system.

What does each option cost in money and effort?

Shared tools cost less to start. Private assistants cost more to set up and need ongoing care. The work that decides success is similar for both: clean documents, clear permissions and a policy staff follow.

Pylon Digital’s Private AI and knowledge assistants are priced as a setup fee plus a monthly fee, both quoted in writing after the free discovery call. Time to launch depends on the number of sources and how much clean-up they need, and is set out in your proposal. After launch, Managed AI runs the assistant: hosting and monitoring, plus seat administration for any shared tools you also use.

Can a firm use both?

Yes, and for many firms that is the practical answer. A shared tool handles general drafting and research that involves no client identifiers. A private assistant handles firm knowledge and client-sensitive work. The acceptable-use policy decides which data goes where.

For example, a 25-person accounting firm might use Copilot for emails, meeting notes and spreadsheets, and a private assistant over its procedures manual, engagement letter templates and prior-year workpapers. Staff know which to open because the policy names the tool for each kind of data.

Which does your firm need? A decision checklist

Answer these eight questions honestly. The pattern of answers points to the right starting point.

Mostly yes to questions 1 to 3: start with a shared tool, set up properly. Any yes to 4 to 7: a private assistant is worth scoping. A mix: use both, with a policy that says which data goes where. If you want a second opinion on your answers, book a free 45-minute discovery call.

This is general information, not legal advice.