Admin-based Authentication

This is the simplest form of crawler authentication. The administrator creates a source, or set of sources. As part of the source definition, the administrator can specify a list of URLs which require authentication, and supply credentials to crawl private content using various authentication mechanisms. It is also the administrator's reponsibility to check the aurhorization policy and make sure that the crawled documents are properly protected.