How often search engine spiders crawl a site. Crawl frequency is influenced by several factors including PageRank, frequency of updates and the number of links to a page. As a rule of thumb, the more often a site is updated, the more frequently crawling will occur.