Advanced Settings

The Advanced Settings allow you more granular control of how Sitebulb crawls and what data it collects.

• Crawler configuration
• Advanced robots settings
• Crawling subdomains
• URL Exclusions
• Ignoring Hints
• Using custom headers
• Specifying language patterns
• Authentication for crawling staging sites
• Adding proxy information
• Send email when audit is completed
• How to control URLs/second for chrome crawler
• Automatically exporting to Google Sheets