Four Ways In, and the Formats We Refuse: How an Agent Learns Your Business

The engine question is the wrong question

Knowledge-base marketing tends to compete on scrapers: this many engines, this one for JavaScript sites, that one for bot-protected pages. It is a satisfying list, and it is not what decides whether your agent answers correctly.

What decides that is narrower. Did the document get in at all? Did the passage containing the answer survive chunking intact? And when retrieval finds nothing, does the agent say so or make something up?

So here is the whole ingestion surface, including the parts we refuse.

Four ways in

Website crawl, one to three levels

Depth is a choice you make rather than a default we hide, and you review the pages before anything is indexed. A crawl that silently swallows your careers page and a 2019 press release is how an agent learns to answer confidently about things you no longer do.

Sitemap import

Reaches what a crawl cannot: unlinked pages, and pages only reachable through your own site search. If your documentation is complete but poorly linked, this is the one you want.

Pasted text

For the policies that live in somebody's head rather than on a page. The real refund window. The thing you always end up telling people who ask about installation. Most businesses have more of this than they expect.

File upload — txt, md, csv and PDF

A PDF with a text layer is read directly — if you can select the words in a PDF reader, so can we. A scanned PDF is read with OCR. A scan is still a picture of text, so check what the agent answers from it before you rely on it for prices.

What we will not take, and why we say so out loud

• Word and Excel files. Export to txt, md or CSV first. It takes ten seconds and it is honest about what we actually parse.
• Images and video. Nothing is extracted from them.
• Google Drive, SharePoint, Confluence. There is no connector for any of them. If the content matters enough to answer customers with, export it. (Notion is different: the agent can query it live as a connected tool, rather than importing it.)

Every one of these is a line we could have blurred on a features page. The cost of blurring it is a customer who uploads forty Word files, sees them accepted, and discovers a month later that the agent never learned a thing from them.

Row-aware CSV chunking

A price list is the commonest thing anybody uploads and the easiest to ruin.

Chunk a CSV by character count and rows split. The product name lands at the end of one passage and the price at the start of the next. Retrieval then hands the model a passage containing a number and no idea what it belongs to — and the model, being helpful, attaches it to whatever product it was asked about.

So a row is never split across passages. A price never separates from its product. It costs nothing to do and it removes the single most embarrassing failure mode in a retrieval system.

What happens when retrieval finds nothing

The similarity threshold was measured against real data rather than guessed at. When a search returns nothing above it, the agent falls back on the business description you wrote — plain facts you gave it — and if that does not cover the question either, it says it does not know.

That last behaviour is the one worth buying, and it is verified rather than hoped for. "Knows your business" is true for the material you added. It is not a claim that the model will improvise the rest.

Ingested text cannot give your agent orders

Everything a tenant adds is fenced and labelled as data before it reaches the model, so a page you crawl cannot smuggle instructions into your agent through a hidden div. That boundary is tested, not asserted: eight live prompt-injection attacks run against the public endpoint as part of the test suite.

One knowledge base, 37 languages

You do not maintain a copy of your knowledge per language. It stays in the language you wrote it, and the model translates as it answers — 37 reply languages, including all 22 scheduled Indian languages, each written in its own script. A Marathi caller and an English visitor are answered from the same source document.

When something fails to index, you are told

Documents that could not be indexed appear in the notifications feed with the reason. A source that silently failed is a question your agent will answer badly for months, and nobody goes looking for a file they believe is already there.

Get started

Point a crawl at your site, pick one to three levels, review the page list, and add the two policies that were never written down anywhere. That is usually eighty per cent of what an agent needs.

Build a knowledge base free →

No credit card required • Trial includes 25 voice minutes and 200 chat replies