Availability and accuracy of distributed web crawlers: A model-based evaluation

M Nasri, S Shariati, M Sharifi - 2008 Second UKSIM European …, 2008 - ieeexplore.ieee.org
M Nasri, S Shariati, M Sharifi
2008 Second UKSIM European Symposium on Computer Modeling and …, 2008ieeexplore.ieee.org
Distributed Web crawlers are extensively used for Web mining nowadays, but their accuracy,
dependability and other operational measures have not been fully studied. Distributed Web
crawlers are costly and require careful selection of configuration parameters. It is important
to have some estimation about the performance, dependability and accuracy of a Web
crawler. This paper presents a model-based evaluation of the accuracy and availability of a
distributed Web crawler whose architecture is based on UbiCrawler. Stochastic activity …
Distributed Web crawlers are extensively used for Web mining nowadays, but their accuracy, dependability and other operational measures have not been fully studied. Distributed Web crawlers are costly and require careful selection of configuration parameters. It is important to have some estimation about the performance, dependability and accuracy of a Web crawler. This paper presents a model-based evaluation of the accuracy and availability of a distributed Web crawler whose architecture is based on UbiCrawler. Stochastic activity networks are used for modelling the crawler. Accuracy and availability of the Web crawler are formally defined, and the effects of environmental failure rates on crawling nodes and on the availability of the whole system are discussed.
ieeexplore.ieee.org
Showing the best result for this search. See all results