Skip to main content
If the answers are already on your website, this is the fastest way to train your bot. Paste a URL and Chatref crawls every page on that domain and learns the text. Want just one page? Paste that specific page’s URL instead.
Your website is already crawled once for you during signup, when you enter it on the “What’s your website?” screen. You only need to add a site here if you want to train on another one – or crawl the same site again after big changes.

​
How to crawl a website

1

Open the Sources tab

Open your agent and go to the Sources tab.
2

Choose Crawl Website

From the add menu, pick Crawl Website.
3

Paste your URL

Enter the address you want to learn from. It must start with http:// or https:// – for example, https://example.com. A homepage URL crawls the whole domain; a deep link like https://example.com/pricing crawls just that page.
4

Queue and train

Submit it. The source is queued and a background worker reads each page. Watch the status until it shows Trained.

​
When to use it

Website training is best when your site already has the answers:
  • A SaaS app – crawl your docs, pricing, and feature pages so the bot can explain how the product works.
  • A dental clinic – crawl your services and hours pages so visitors get details without calling.
  • A travel agency – crawl your destination and FAQ pages in one go.
Start with your homepage URL to capture the whole site at once. If you only want the bot to learn one thing – say, a single policy page – paste that page’s exact URL instead.

​
Next steps

Add files

Upload documents your website doesn’t cover.

Add text

Paste an answer that isn’t on your site yet.

Track training

See when the crawl finishes and retry if it fails.

Test your bot

Ask your bot a question in the Playground.