Lam, Wang and Garcia-Molina, Hector (2001) Multicasting a Changing Repository. Technical Report. Stanford.
Web crawlers generate significant loads on Web servers, and are difficult to operate. Instead of repeatedly running crawlers at many "client" sites, we propose a central crawler and Web repository that multicasts appropriate subsets of the central repository, and their subsequent changes, to subscribing clients. Loads at Web servers are reduced because a single crawler visits the servers, as opposed to all the client crawlers. In this paper we model and evaluate such a central Web multicast facility for subscriber clients, and for mixes of subscriber and one-time downloader clients. We consider different performance metrics and multicast algorithms for such a multicast facility, and develop guidelines for its design under various conditions.
|Item Type:||Techreport (Technical Report)|
|Uncontrolled Keywords:||Web repository WebBase multicast facility scheduling|
|Subjects:||Computer Science > Databases and the Web|
|Related URLs:||Project Homepage||http://www-diglib.stanford.edu/diglib/pub/|
|Deposited By:||Import Account|
|Deposited On:||11 Nov 2001 16:00|
|Last Modified:||27 Dec 2008 10:21|
Available Versions of this Item
- Multicasting a Changing Repository. (deposited 11 Nov 2001 16:00) [Currently Displayed]
Repository Staff Only: item control page