Adding sources
Sources give your AI assistant the material it uses to answer questions. Add resources lets you discover web pages and online documents, upload files, or connect Google Drive and RSS.
1. Open Add resources
In the Knowledge base, click Add resources. The dialog has Web, Files, Google Drive, and RSS tabs. Detailed options are collapsed under Advanced settings.
Document usage appears at the top. Agent access sits beside the tabs, or above them on smaller screens:
- In the organization's knowledge base, All agents (including new) is selected by default.
- In an individual agent's knowledge base, that agent is preselected. You can change the selection here too.
- All agents is the first dropdown option. Selecting an individual agent turns it off; you can select multiple individual agents.
Access inherited from a parent folder still applies. See shared source access.
2. Find web sources
- On Web, enter a website, a section of a website, or a direct document URL such as a PDF.
- Optionally open Advanced settings and choose a discovery method. Quick discovery is the default for ordinary websites; use Find documents in listings for a page listing PDFs.
- Click Search or press Enter. Loading appears in the button and results table, with elapsed time above the table spinner.
- Review the results and select sources to import. Searching alone does not add them to the knowledge base.
- Click Add in the bottom bar. It shows the selected count and projected document usage. Reduce your selection if it exceeds the limit.
Sources enter the processing queue. The assistant can use them after processing finishes. A direct document URL uses the single-file import flow; scheduled discovery of other sources does not apply to it.
Filtering is different from selecting sources
The text filter shares a row with Pages and Documents checkboxes, both enabled by default. Counts show the discovered sources of each type.
Turning off a filter only hides those resultsâit does not deselect them for import. Select all and Deselect all affect only currently visible results matching both the text filter and enabled types. To exclude an archive, for example, filter its URLs and deselect the displayed sources.
URL parameters and fragments after # are preserved. They can identify a language variant, a section of a page, or different content. Filter and deselect unwanted variants. The same document can be linked from multiple places; review the selection before importing.
Repeating a search and handling errors
After changing the URL or discovery settings, use Search again. Each search makes a new request rather than reopening a cached result. Errors appear in the table; retry with the same button.
A partial result is not a complete website inventory. Depth, limits, or unavailable pages can restrict it. Read the status message and adjust the scope if needed. An empty result does not necessarily mean there are no documents: some pages load slowly or restrict automated access.
3. Discovery methods in Advanced settings
Each method has help directly in the dropdown.
| Method | When to use it | What discovery does |
|---|---|---|
| Quick website scan â default | An ordinary website and a quick first selection | Finds URLs through the website map. It may miss documents not listed there. Content is processed after import. |
| Find documents in listings | Notice boards, regulation lists, or file directories | Reads listing pages and collects links to supported documents without reading every file. Usage depends on the number of visited pages. |
| Follow links and scrape | When following links is needed and quick discovery is insufficient | Visits and reads resources during discovery, including supported documents. Usually costs more; PDF parsing is charged per page. Selected sources are also processed during import. |
| This URL only | One specific page | Adds only the submitted URL. Does not follow links or find attachments. Content refresh remains available. |
Listing discovery also includes visited listing pages as potential sources. Deselect them if you do not want their content imported.
Depth and limits
- Maximum depth applies to listing discovery and link crawling. The default is 5, with a range of 0â100. Depth 0 visits only the starting page; listing discovery still collects its document links, but does not open other listings or pagination pages.
- Quick discovery has Maximum URLs returned: default 5,000, maximum 100,000 returned URLs.
- Listing discovery and crawling have Maximum pages visited: default 200, maximum 10,000 visited pages. This does not guarantee the same number of discovered files.
These settings bound discovery. They do not replace your plan's document limit.
Limiting discovery to part of a website
The simplest way: enter the section's address
Most of the time you do not need any rules. The address you enter in the Web field already sets where Kanbu looks. Kanbu stays on the same website and covers only the page you entered and everything below it.
| You enter in the Web field | Kanbu covers | Kanbu skips |
|---|---|---|
https://www.example-school.com | the whole website | nothing |
https://www.example-school.com/news | /news and everything below it, such as /news/new-information-system | /contact, /about, /after-school-club |
A trailing slash makes no difference. The boundary applies to whole path segments: /news includes /news/⌠but not /news-archive.
So to add only the news from a school website, paste the address of the news page and click Search. Leave Include paths empty.
When to use rules (regex)
Rules help when the address alone is not enough, for example when you want to:
- leave out part of a section (an archive, an English version),
- pick several separate sections from one website,
- skip certain file types.
Use Include paths (regex) and Exclude paths (regex) under Advanced settings. They accept regular expressions: short patterns that describe what an address should look like. Put one rule on each line. An empty field means "no restriction".
How the rules work:
- Only the path after the domain is compared. For
https://www.example-school.com/news/new-information-system?page=2that is/news/new-information-system. The domain and anything after?are not used. - A rule matches if it fits anywhere in the path. The rule
newsmatches/newsbut also/archive/news-2020. To be precise, use^for the start of the path and$for its end. - Include: when the field has rules, an address must match at least one of them. An empty field lets everything through.
- Exclude: an address that matches any rule is skipped. Exclusions always win over includes.
- Matching is case-sensitive. To ignore case, start the rule with
(?i). - Accented characters in addresses are encoded.
/ĹĄkolais stored as/%C5%A1kola. Prefer parts of the address without accents. - Lookaround and backreferences are not supported.
Handy patterns
| I want to | Field | Write |
|---|---|---|
| only the News section and everything below it | Include | ^/news(/.*)?$ |
only addresses starting with /documents/ | Include | ^/documents/ |
| skip the archive | Exclude | ^/news/archive |
| skip the English version of the site | Exclude | ^/en/ |
| skip ZIP files | Exclude | (?i)\.zip$ |
| two separate sections | Include | ^/news and on the next line ^/documents |
Include paths and the starting page
Include rules also apply to the page you entered and to the listing pages Kanbu passes through on the way to further links. If they do not allow it, the search fails with The starting URL must match the include/exclude patterns. With listing discovery, a rule that lets through only addresses ending in .pdf would also block the listing pages that lead to the files.
Example: you enter https://www.example-school.com in the Web field and write ^/news/ in Include paths.
- The starting page's path is
/, which does not match^/news/, so you get the error. - The page
/newsitself (no trailing slash) would not match the rule either.
There are two ways to fix it:
-
Enter the section's address instead of the whole website:
https://www.example-school.com/news. You can then leave Include paths empty. This is the simplest fix. -
Add rules that allow both the starting page and the section:
^/$ ^/news(/.*)?$The first line allows the website's home page. The second allows
/newsand everything below it.
4. Hierarchy and ongoing updates
Advanced settings contains separate controls:
| Setting | Purpose |
|---|---|
| Look for new sources | How often to discover new URLs. Default: Never. |
| Automatically include new sources | Appears after a discovery frequency is enabled. Off by default: new results wait for your selection. |
| Auto Refresh | How often to check the content of sources already added. Default: Inherit, using the parent-level setting. |
| If Source Removed | Flag, remove, or ignore a confirmed unavailable source according to its policy. Standalone sources inherit the folder policy; outside a folder, they are flagged for review. |
| Mirror website hierarchy | Organizes imported sources into folders matching website paths. Can also be enabled without scheduled discovery. |
Scheduled discovery enables hierarchy mirroring and locks its switch. Disabling discovery restores your manual hierarchy choice. This URL only does not use scheduled discovery or hierarchy mirroring.
A tracked website saves the method, depth, limit, and path rules for future scans. An ordinary one-off import without a tracked folder does not save a discovery schedule.
Discovery and content refresh have different usage costs. Follow links and scrape reads content during discovery even when Auto Refresh is off. Recurring automatic checks depend on your plan's permissions. See Updating sources and Website monitoring.
5. Files, Google Drive, and RSS
- Files: choose local files or drag them into the upload area. Check the upload field for supported formats; multiple files can be added.
- Google Drive: enter a public folder link and confirm beside the URL. Subfolders are included by default. Options and updates are under Advanced settings. Google Drive instructions.
- RSS: enter an RSS or Atom feed URL and confirm beside it. Advanced settings controls how items are read and refreshed. RSS instructions.
Product XML feeds for e-commerce are a separate feature. See E-commerce agent.
After adding sources
Check processing status in the knowledge base. Parsing means reading the source, Embedding prepares it for retrieval, and Done means the content is ready. If processing fails, check the source's availability and format; retry through Refresh data. Duration depends on source count and size. Once an import is queued, you can close the app while processing continues in the background.
Manage a source through its row menu or bulk actions. Deleting a source affects every agent using it. To remove only one agent's access, change access instead of deleting the source.