36.1 Purpose of Auto-Index Visited Pages

Auto-index visited pages allows an Agent to discover website content through real visitor page activity.

When enabled, eligible new visitor pages can be queued for crawling and indexing so the Agent's Knowledge can grow automatically.

36.2 Finding the Setting

1. Open the Agent.
2. Select Settings.
3. Locate Auto-index visited pages in the Launch/Settings area.

36.3 Enabling Auto-Indexing

Select the Auto-index switch to turn the feature On.

The setting is saved directly when toggled. It is not a separate manual Source-add workflow.

36.4 What Happens After It Is Enabled

As visitors reach eligible pages using the Agent widget, AIUNIFY Chat can queue those pages for background crawl and indexing.

This can reduce the need to manually enter every newly discovered page as an individual Knowledge Source.

36.5 Automatic Safety Skips

The deployed interface states that Auto-Index automatically skips common sensitive or transactional paths including:

• /admin
• /checkout
• /cart
• /account
• /login

36.6 Rate Guardrail

The Settings page also states that automatic indexing is capped at 30 pages per hour per Agent.

This prevents uncontrolled page discovery from creating an unlimited crawl queue.

36.7 Auto-Indexing and Restricted Paths

Restricted Paths and Auto-Index safety rules serve related but different purposes. Restricted Paths control where the widget mounts. Auto-Index safety rules help determine which visited pages should be considered for automatic Knowledge ingestion.

36.8 Reviewing Automatically Indexed Content

Do not assume every automatically discovered page is good Knowledge. Periodically review the Agent's Knowledge and Sources to confirm that indexed content is relevant and usable.

36.9 When to Disable Auto-Indexing

Consider turning Auto-Index off when:

• The website contains many pages that should not become Agent Knowledge.
• You require tightly curated Knowledge Sources.
• Website content changes in ways that require manual review before indexing.

36.10 Operational Best Practice

Use Auto-Index as a controlled discovery tool rather than as a replacement for Knowledge management. Monitor what the Agent learns, and remove or reindex Sources when needed.